Add code-grounded brainstorm for data-provider and accounting improvements - #2100
Conversation
…ments Documents 10 improvement lanes from a fresh exploration of the provider and accounting subsystems: streaming base-class unification, canonical symbol resolution, unified data quality, backfill feedback, provider failure hardening, mark-to-market wiring, automated journal drafts, closing entries, ledger spine unification, and a fill-to-ledger durability fix. Records the session in the brainstorm history ledger. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PdMuw9yNjY451tw5VEordn
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request adds a comprehensive brainstorm document (docs/product/data-provider-accounting-brainstorm-2026-07.md) and updates the brainstorm history log (.claude/skills/meridian-brainstorm/brainstorm-history.jsonl). The document outlines ten grounded improvement lanes for the data provider and accounting subsystems, addressing structural fragmentation in streaming providers and the integration gap between the Meridian.Ledger projector library and the live ILedgerJournalStore path. There are no review comments provided, so I have no feedback to evaluate.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Runs the core docs automation profile so the doc-health dashboard, example-validation scan, and repository structure include the new brainstorm document, satisfying the regenerate-docs CI check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PdMuw9yNjY451tw5VEordn
…accounting-brainstorm-o73lsw # Conflicts: # docs/status/doc-health-dashboard.json # docs/status/doc-health-dashboard.md
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c9eedd00ae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| "docs/plans/ufl-warrant-target-state-v2.md", | ||
| "docs/plans/wave-implementation-checklists.md", | ||
| "docs/plans/web-ui-development-pivot.md", | ||
| "docs/product/data-provider-accounting-brainstorm-2026-07.md", |
There was a problem hiding this comment.
The generated health dashboard now classifies the added docs/product/data-provider-accounting-brainstorm-2026-07.md as an orphan, and docs/product/README.md is the canonical stakeholder-facing entrypoint that already links the other July brainstorms. In the stakeholder/docs-health workflow, this means the new product guidance is effectively undiscoverable and intentionally increases the orphan count; add it to the product README or another indexed product page so the generated report no longer flags it as orphaned.
Useful? React with 👍 / 👎.
Addresses the review note that the new brainstorm document was classified as an orphan: adds it to the product README's stakeholder reading list and regenerates the doc-health dashboard. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PdMuw9yNjY451tw5VEordn
Refreshes the doc-health dashboard, TODO scan, coverage report, example-validation scan, and repository structure so the generated outputs match the merged tree, satisfying the regenerate-docs check. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PdMuw9yNjY451tw5VEordn
…accounting-brainstorm-o73lsw
The regenerate-docs check runs on the PR merge ref, so the generated dashboards must reflect current main. Merges main (through PR #2124's workstation route consolidation) and refreshes the coverage report, API contract dashboard, governance readiness dashboard, and TODO scan to match. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PdMuw9yNjY451tw5VEordn
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3105931e76
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| user-visible cost is real: only base-class providers emit `IProviderConnectionDiagnosticsSource` | ||
| diagnostics, so `ProviderEndpoints.cs` (in `src/Meridian.Ui.Shared/Endpoints/`) falls back to | ||
| reporting `IsConnected` from `s.Enabled` — the Data workspace can show a provider as connected | ||
| when it has merely been configured. |
There was a problem hiding this comment.
This premise is stale in the reviewed tree: NyseMarketDataClient, RobinhoodMarketDataClient, and IBMarketDataClient all implement IProviderConnectionDiagnosticsSource, and ProviderEndpoints consults ProviderConnectionDiagnosticsProjection before falling back to enabled/metrics state. Keeping this as a headline gap sends implementers toward an unnecessary base-class migration instead of the real provider-status gaps.
Useful? React with 👍 / 👎.
| `SymbolMappingService`. Worse, the resolver is wired inconsistently: `ProviderFactory` and | ||
| `BackfillCoordinator` pass it into `CompositeHistoricalDataProvider`, but | ||
| `BackfillWorkerService.cs:759` constructs the composite **without any resolver**, so the | ||
| background backfill path silently does no cross-provider symbol translation. And the composite's | ||
| opt-in cross-validation (`ValidateBarsAsync`) calls the validation provider with the *unresolved* | ||
| symbol — guaranteed false discrepancies against providers like Stooq that need `aapl.us`. |
There was a problem hiding this comment.
This no longer matches the code under review: BackfillServiceFactory.CreateServices builds an OpenFigiSymbolResolver when EnableSymbolResolution is true and passes it into CompositeHistoricalDataProvider, and ValidateBarsAsync resolves the validation provider symbol before fetching. Leaving this backlog item in the brainstorm directs work at a bug that has already been fixed.
Useful? React with 👍 / 👎.
| none share a score. Meanwhile the surface asymmetry is stark: ~35 REST endpoints exist in | ||
| `DataQualityEndpoints.cs` and WPF has a full `DataQualityPage`, but the browser workstation has | ||
| **no data-quality dashboard component at all**. | ||
|
|
||
| The work: define one shared quality read-model (per-symbol composite health drawing from all | ||
| three sources — streaming freshness, stored completeness, adapter-detected gaps) in | ||
| `Meridian.Ui.Shared`, adapt the three subsystems to feed it, and build the React Data Quality | ||
| view in `src/Meridian.Ui/dashboard/` on the endpoints that already exist. This is the W1 |
There was a problem hiding this comment.
The browser workstation already has this surface in the reviewed commit: src/Meridian.Ui/dashboard/src/screens/data-screen.tsx renders DataQualityRegion, backed by data-screen.data-quality.view-model.ts and tests, over /api/quality/dashboard. Describing it as having no dashboard component will send roadmap work toward duplicating an existing React panel instead of improving the current one.
Useful? React with 👍 / 👎.
| The single highest-leverage accounting finding: `DailyPortfolioPricingProjector` | ||
| (`src/Meridian.Ledger/`) — policy-driven fair-value marks producing balanced unrealized-P&L | ||
| journal lines with price-source evidence, fully covered in `LedgerIntegrationTests` — has **no | ||
| live caller**. The live path (`LedgerPostingConsumer` in `src/Meridian.Execution/Events/`) posts |
There was a problem hiding this comment.
This highest-leverage item is based on an obsolete premise: src/Meridian.Application/Accounting/DailyMarkToMarketService.cs already calls DailyPortfolioPricingProjector.Project, builds a governed draft, and is covered by DailyMarkToMarketServiceTests. The remaining work may still be broader integration, but saying the projector has no live caller misstates current accounting capability.
Useful? React with 👍 / 👎.
| Small, sharp, and arguably a bug: `LedgerPostingConsumer` | ||
| (`src/Meridian.Execution/Events/LedgerPostingConsumer.cs`) documents that trade events "are never | ||
| silently discarded" and configures its bounded channel with `FullMode = Wait` — but its `Publish` | ||
| method uses non-blocking `TryWrite` and merely logs a warning when the channel is full. Under | ||
| backpressure, executed fills are dropped **before they reach the books**. For an |
There was a problem hiding this comment.
In this commit, LedgerPostingConsumer.Publish already takes the slow path that waits with WaitToWriteAsync() until the channel can accept the fill, and EventDrivenDecouplingTests includes LedgerPostingConsumer_WhenChannelIsFull_BlocksPublisherInsteadOfDroppingFills. Keeping this as a current high-impact bug misdirects readers toward fixing behavior that is already present.
Useful? React with 👍 / 👎.
Main implemented several brainstorm items within a day of writing: provider connection diagnostics, backfill worker symbol resolution, the browser data-quality panel, typed SLA metadata with progress events, DailyMarkToMarketService, and the blocking fill publish path. Adds a per-idea status table and dated inline corrections at each stale premise so the document directs remaining work accurately, addressing the five Codex review comments. Refreshes generated doc inventories for the size change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PdMuw9yNjY451tw5VEordn
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bd9a64e55a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | Shared Contracts | Contract compatibility matrix requires review packets and owner decisions | Gap | 0/3 | `docs/status/contract-compatibility-matrix.md` | terms: `Contract review packet`, `Owner decision`, `migration notes` | | ||
| | Governance Operations | Feature inventory describes reconciliation calibration and sign-off posture | Gap | 0/2 | `docs/status/FEATURE_INVENTORY.md` | terms: `calibration-summary`, `tolerance-profile posture`, `required sign-off role` | | ||
| | Governance Operations | Workstation endpoints expose governance break queue and calibration routes | Pass | 3/3 | `src/Meridian.Ui.Shared/Endpoints/WorkstationEndpoints.cs` | - | | ||
| | Governance Operations | Workstation endpoints expose governance break queue and calibration routes | Gap | 0/3 | `src/Meridian.Ui.Shared/Endpoints/WorkstationEndpoints.cs` | terms: `calibration-summary` | |
There was a problem hiding this comment.
This generated readiness row is now a false gap: the endpoint is still registered in src/Meridian.Ui.Shared/Endpoints/WorkstationEndpoints.cs:1790 via UiApiRoutes.ReconciliationCalibrationSummary, and endpoint coverage remains in tests/Meridian.Tests/Ui/WorkstationEndpointsTests.cs. Leaving the dashboard at 0/3 makes governance readiness look regressed even though the route is present; update the generated .md/.json evidence to recognize the constant-backed route instead of requiring the literal calibration-summary text.
Useful? React with 👍 / 👎.
| | 4 | Backfill feedback loop | Done | `OnProgressUpdate` is raised with subscriber-failure guards; SLA metadata is typed (`BackfillRemediationSlaDecision`/`Metadata`/`Status`); the remediation provider is an options default rather than hard-coded. | | ||
| | 5 | Failure & rate-limit hardening | Partially done | `DataSourceRegistry` now records activation/registration failures via `RecordFailure`. Remaining: typed rate-limit detection (string matching on `"429"`/`"rate limit"` still present in the composite), streaming-side rate-limit tracking, and confirming registration failures surface in the provider catalog UI. | | ||
| | 6 | Mark-to-market wiring | Largely done | `DailyMarkToMarketService` runs `DailyPortfolioPricingProjector` into governed drafts, with tests. Remaining: `NavAttributionService` still computes `totalNav` as a sum of all component balances rather than assets − liabilities, and NAV consumption of the marked book needs end-to-end verification. | | ||
| | 7 | Automated journal drafts | Partially done | Corporate-action/dividend event producers and draft intake shipped (`AutomatedJournalEventKind`, `DurableAutomatedJournalPoster`, `AutomatedJournalDraftIntakeService`). Remaining: management/performance-fee and withholding-tax accrual producers on a schedule. | |
There was a problem hiding this comment.
In the current tree, management/performance fee accrual producers are already implemented and wired: FeeScheduleAccrualEventProducer emits both event kinds, AutomatedJournalIntakeRunner.RunFeeAccrualIntakeAsync lands them in the workbench queue, and LedgerEndpoints exposes the fee-accrual intake route. Saying those producers remain will send accounting work toward duplicating an implemented close-cockpit intake path; narrow the remaining work to scheduling and any still-missing withholding flow.
Useful? React with 👍 / 👎.
| ### 4. Backfill Feedback Loop: Live Progress and Typed SLA Metadata | ||
|
|
||
| Two verified pieces of unfinished plumbing blunt the backfill experience. First, | ||
| `CompositeHistoricalDataProvider.OnProgressUpdate` is declared and documented but **never raised** |
There was a problem hiding this comment.
The current code now raises progress through CompositeHistoricalDataProvider.RaiseProgress, stores AutoRemediationSla as typed BackfillRemediationSlaMetadata, and uses AutoGapRemediationPolicy.DefaultProvider rather than a hard-coded remediation call-site. This paragraph still says progress is never raised, SLA state is only string warnings, and the provider is hard-coded, so readers following the lane will try to fix completed work despite the status table marking it done.
Useful? React with 👍 / 👎.
Summary
Documentation-only change: adds
docs/product/data-provider-accounting-brainstorm-2026-07.md, a Domain-Focused brainstorm of 10 concrete, code-grounded improvement lanes for the data-provider and accounting subsystems, and records the session in.claude/skills/meridian-brainstorm/brainstorm-history.jsonl. No runtime behavior changes.Headline findings the document is built on (from a fresh code exploration):
Meridian.Ledgerprojector library (daily pricing, amortization, tax-lot relief, multi-currency, partnership waterfalls, automated fee/dividend drafts, shadow NAV) is fully tested but has zero live callers — so the live path posts securities at cost and NAV is effectively NAV-at-cost.WebSocketProviderBase(so only they emit connection diagnostics), symbol identity lives in three disconnected mechanisms (withBackfillWorkerServiceconstructing the composite without a symbol resolver), and three overlapping data-quality subsystems don't share a score.LedgerPostingConsumerdocuments "never silently discarded" fills but publishes with non-blockingTryWrite, dropping fills under backpressure.The document includes a summary table, per-idea narratives with codebase anchors and tradeoffs, and a sequencing recommendation (quick correctness wins first, mark-to-market wiring as the marquee item, spine unifications last).
Phase declaration:
PR9— the diff is docs plus the brainstorm-skill session ledger (.claude/skills/**), which no docs-scoped phase covers; matches the precedent of PR #2078.Reason
Requested brainstorm of concrete, highly valuable improvements to the data-provider- and accounting-related pieces of the codebase, executed via the
meridian-brainstormskill workflow (context gathering, mode declaration, idea narratives, synthesis, history ledger update).Testing performed
bash scripts/ci.shcompleted successfullyquality-gatepassedNot run: this PR adds a Markdown document, an index link, and a JSONL ledger line only; no source, test, or build files are touched. Generated doc inventories were refreshed with
run-docs-automation.py --profile core(the same command theregenerate-docscheck runs).Safety review
mainmainwas performedGovernance changes
Check every governance file modified:
.github/workflows/**.github/CODEOWNERS.github/pull_request_template.mdAGENTS.mdscripts/ci.sh🤖 Generated with Claude Code
https://claude.ai/code/session_01PdMuw9yNjY451tw5VEordn