db/state, execution: reconcile StateCache and BranchCache at the file-publication boundary - #23047
Draft
yperbasis wants to merge 2 commits into
Draft
Conversation
…the file-publication boundary Snapshot downloads publish state that never flows through cache applies. The PR-23033 reconciliation ran at one caller (ProcessFrozenBlocks) on its normal exits only: an error between publication and the absorb skipped it while Start logs and continues, and the commitment BranchCache — whose Get guards only against unwind staleness — was not reconciled at all, so a branch warmed before a download could restore the trie to the wrong state. Move reconciliation to the chokepoint every publication funnels through: recalcVisibleFiles absorbs the bound StateCache (per-domain values-files ends, no tx involved) and the aggregator-owned BranchCache, under dirtyFilesLock and before the new bundle is published, so readers never see extended files while stale entries live. BranchCache gains the applied-watermark it lacked (putWatermark, bumped on every Put): own-built files are a no-op, foreign state clears. BindAggregator's duck becomes BindStateCache — the aggregator holds the cache and reconciles immediately at bind, covering wired-after-files-visible ordering; the PFB-local absorb is removed as superseded. Error paths and mid-run downloads are covered by construction. Closes #23028.
The watermark test was written with the boundary-hook commit but never staged (git add -u skips new files); it sat untracked, breaking compilation on sibling branches.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #23028.
Stacked on #23033 (draft until it merges, then retargets to
main).Snapshot downloads publish state that never flows through cache applies, so nothing overwrites the entries they invalidate and
appliedEndknows nothing happened. #23033 reconciles theStateCacheat one caller —ProcessFrozenBlocks, on its normal exits. Review (codex) found the two gaps that placement leaves, and this PR moves reconciliation to the event itself.The two gaps
RunSnapshotswith fallible work after it; an error there returns before the absorb, andExecModule.Startlogs and continues — published files then coexist indefinitely with stale cache entries.main, not a execution, db: keep StateCache coherent through startup catchup; batch applies #23033 defect):BranchCache.Getguards only against unwind staleness (epoch + floor), and a download advances neither — a branch warmed before startup catchup (engine endpoints are live beforeStart) can restore the trie from a dead checkpoint and corrupt the first post-snapshot commitment root. Likely backport-relevant (release/3.6 has both the BranchCache and snapshot downloads).The fix: reconcile where publication happens
recalcVisibleFilesis the chokepoint every publication funnels through (OpenFolder,ReloadFiles, merges, dependency toggles) — the visibility-lowering assert already lives there. It now also reconciles, underdirtyFilesLockand before the new bundle is published, so readers never observe extended files while stale entries live. No caller and no error path can skip it; mid-run downloads are covered for free — which is what closes #23028 rather than just its startup instance.BindAggregator's duck contract becomesBindStateCache(*StateCache)— the aggregator holds the cache (legal:execution/cachehas no dependency ondb/state) and absorbs per-domain values-files ends at every recalculation, plus immediately at bind (files may already be visible when the cache is wired). The frontier needs no transaction, which removes the tx-liveness problem the caller-side placement had. Built-from-applied files stay at or belowappliedEnd, so the every-merge path is a strict no-op. execution, db: keep StateCache coherent through startup catchup; batch applies #23033'sProcessFrozenBlocks-local absorb is removed as superseded.putWatermark, the exclusive txNum end of the trie's own writes, bumped on everyPut.AbsorbFilesExtension(filesEnd)clears everything only when files exceed it: own-built commitment files are a no-op, downloaded state clears. Reconciled unconditionally (the BranchCache is aggregator-owned and commitment correctness is not optional).Lock order is safe: the hook runs under
dirtyFilesLockand takes the caches' internal locks; no path takes them in the reverse order (fills resolve frontiers via atomic bundle loads, applies take onlyadmissionMu).Testing
Red-first: an aggregator-level publication test (fill a state entry and warm a branch from pre-publication views,
generateStateFiles+OpenFolder— both caches must drop their entries and reject pre-publication refills); a bind-after-files-visible test (the admission frontier starts at the published ends); and aBranchCachewatermark unit test (own-files no-op, foreign-files clear, no re-clear). Suites: fulldb/state,execution/commitmentunder-race(95s),execution/cache+db/state/execctxunder-race, execmodule + tester, stagedsync, and the engineapi boot tests (full node boot throughOpenFolderwith a bound cache — the lock-order claim exercised end to end).make lintclean.