execution, db: bind cache views to database and file generations

[yperbasis]

Fixes #22463.
Fixes #23028.

Problem

StateCache and BranchCache are process-global caches of latest state. A
transaction may outlive a canonical commit, unwind, or immutable-files
publication, so the caches must reject reads and fills from transactions backed
by older state.

The previous coherence mechanisms did not identify that complete backing state:

• An unwind can discard values that are not present in its callback diff. An
  old transaction could therefore read or refill a dead-fork value after the
  unwind.
• Snapshot downloads can extend the visible domain files without advancing
  PlainStateVersion and without sending the downloaded values through cache
  applies. Existing positive or negative entries could remain live after their
  backing files changed.

Cache generation

Both caches now follow one invariant:

A shared cache represents one durable PlainStateVersion over one compatible
immutable-files view. A transaction can use the cache only while that exact
generation remains published.

A StateCache generation contains the state version and the exclusive values-file
ends for accounts, storage, and code. A BranchCache generation contains the
state version and the commitment values-file end.

DomainVisibleEnd still decides whether a transaction has an exact domain read
frontier. It is not used as the files identity because it describes the combined
database-and-history frontier and can remain unchanged when the pinned values
files change. TxNumsInFiles provides the values-file ends needed to identify
that part of the transaction's backing state.

Each published generation also has an immutable pointer token. Pointer identity
lets publication revoke readers before a new durable generation exists and
prevents a previously revoked view from becoming valid if the same numeric
state version appears again.

Canonical publication

SharedDomains.Commit publishes StateCache and BranchCache through the same
durable sequence:

1. Flush changes into the database transaction and collect cache updates.
2. Prepare adaptive branch-cache changes without mutating the shared cache.
3. Revoke the current cache generations and wait for in-flight fills.
4. Commit the database transaction.
5. Apply the collected updates and publish generations with the committed state
   version.

If the database commit fails, publication restores the previous tokens and
leaves cache entries unchanged.

Reads check their token before and after the underlying lookup. A concurrent
publication therefore turns the result into a miss. Fills recheck the token
while serialized against publication, so an old transaction cannot populate a
new generation.

Unwind behavior

A speculative or locally rewound SharedDomains detaches from both shared
caches. It cannot read, fill, clear, or revoke them.

After a canonical unwind commits, both caches are cleared and published at the
resulting state version. The full clear is intentional: an unwind diff is not a
complete inventory of entries that may have entered either cache from the
discarded fork.

Files publication

Aggregator.recalcVisibleFiles is the common publication boundary for
downloads, reopen, merge, and dependency recalculation. It now reconciles both
caches before the new immutable-files bundle becomes visible.

The caches track how far their committed updates cover each relevant domain:

• Files covered by committed updates retain their entries.
• A files extension beyond that coverage contains state that did not pass
  through this process's cache publication stream, so the affected cache is
  cleared.
• Any change of the relevant file ends revokes existing views, even when
  entries can be retained.
• Publication remains blocked until the new files bundle and its matching cache
  generation become visible together.

A transaction pinned to the previous files cannot bind to the cache after this
boundary: its file ends do not match the newly published generation. Binding a
StateCache after files are already visible performs the same reconciliation.
Visibility lowering remains forbidden while the shared caches are attached.

Simplification

The shared generation rule replaces the StateCache read-view epoch and the
per-entry unwind epochs, floors, and transaction numbers in both caches. The
separate execution/cache/coherence package is removed.

TxNum remains on transient committed updates only to advance file-provenance
watermarks. Cached state entries still keep their source Step, which callers
need independently of cache coherence.

Performance

The cache-hit path remains lock-free and allocation-free. It adds an atomic
pointer comparison before and after the existing lookup. A fill performs an
early pointer check and one authoritative check under the admission lock.

Transaction state versions and file ends are checked when cache views are
constructed, not on each lookup. Long flush and adaptive planning work happens
before publication starts, keeping the read blackout limited to the database
commit and cache publication window.

Forward commits retain unchanged entries. Canonical unwinds and foreign files
extensions clear their affected caches and pay the re-warm cost.

One deliberate trade-off is that a long-lived transaction loses cache reads
after another transaction publishes a new generation. Readers miss instead of
waiting or accepting newer cache state.

Tests

The changes were developed with red-green-refactor regression tests. Coverage
includes:

• staged-unwind and pre-unwind reads and fills;
• old database transactions attached after an unwind;
• old-file transactions first attached after a files publication;
• exact generation matching, publication abort, reset, and atomic visibility;
• StateCache and BranchCache publication after commit;
• speculative unwind isolation and canonical unwind clearing;
• failed database commits restoring both generations;
• adaptive branch-cache planning and publication;
• foreign files invalidation and locally covered files retention;
• binding after files are already visible;
• publication remaining blocked until the new files bundle is visible;
• commitment-domain visibility-lowering protection.

Local verification:

• go test ./execution/cache ./execution/commitment ./db/state/execctx ./execution/exec -count=1
• go test ./db/state -count=1
• go test -race --timeout 20m ./execution/cache ./execution/commitment ./db/state/execctx ./execution/exec ./db/state -count=1
• make lint repeatedly
• make erigon integration

[Copilot]

Pull request overview

This PR refactors execution/cache so StateCache is published and consumed strictly as a single durable-state snapshot keyed by PlainStateVersion (state version), replacing the prior mix of applied-frontier tracking plus per-entry epoch/floor coherence. The goal is to close stale-fill windows around unwinds/commits by making cache usability contingent on an exact state-version match, and by revoking read views before any publication mutates cache contents.

Changes:

• Introduce a version-bound cache.ReadView/publication token model (Publisher/Publication) and remove StateCache per-entry epoch/floor + applied-frontier coherence.
• Rewire canonical vs speculative cache ownership (SetCanonicalStateCache vs SetStateCacheReader) and update execution paths (FCU, SetHead, integration runner) accordingly.
• Update read-ahead warmup and tests/benchmarks to use state-version-bound cache views and new step-based cache APIs.

Reviewed changes

Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
execution/vm/contract.go	Update jumpdest cache usage to new Put signature.
execution/execmodule/set_head.go	Switch SetHead path to canonical cache publication API.
execution/execmodule/forkchoice.go	Switch FCU canonical contexts to canonical cache publication API; remove warmup draining.
execution/execmodule/exec_module.go	Remove read-ahead draining helper; use reader-only cache attachment in ValidateChain.
execution/exec/blocks_read_ahead.go	Bind read-ahead warmup fills to PlainStateVersion and domain visibility; fill using kv.Step.
execution/exec/blocks_read_ahead_test.go	Adapt warmup tests to version-bound ReadView and publisher initialization.
execution/cache/view.go	Redefine ReadView as a durable-state-version handle with generation checks around reads/fills.
execution/cache/state_cache.go	Implement generation token, Publisher/Publication, step-based updates, and version-gated View.
execution/cache/generic_cache.go	Remove unwind coherence from GenericCache; store (value, step) for domain caches.
execution/cache/generic_cache_concurrency_test.go	Update concurrency tests for new Put/PutIfAbsent signatures and semantics.
execution/cache/code_cache.go	Remove unwind coherence; make code layers generation-cleared; switch stamping from txNum/epoch to step where needed.
execution/cache/code_cache_concurrency_test.go	Update concurrency tests for new code-cache semantics (clear-fencing only).
execution/cache/code_cache_codehash_test.go	Replace unwind-based tests with clear/publication-based expectations; update APIs.
execution/cache/cache.go	Update Cache interface to step-based API and simplify package-level docs.
execution/cache/cache_test.go	Update StateCache/DomainCache/CodeCache tests to publisher-based publication model and step API.
db/state/execctx/statecache_rpc_integration_test.go	Add/adjust integration tests covering unwinds and RPC views under version-bound cache publication.
db/state/execctx/statecache_readfill_test.go	Rewrite read-fill/unwind tests to new inactive-during-publication behavior and version-bound views.
db/state/execctx/statecache_readfill_bench_test.go	Remove frontier memo benchmark; align benchmark description with generation-check model.
db/state/execctx/flush_storage_cache_test.go	Update storage-cache commit callback test to read via current version-bound cache view.
db/state/execctx/export_test.go	Adjust exported test helpers to use version-bound cache view wiring.
db/state/execctx/domain_visible_end_memo_test.go	Remove now-obsolete visible-end memo concurrency tests.
db/state/execctx/domain_shared.go	Replace frontier memo + applier model with version-bound view selection and canonical Publisher publication pipeline.
db/state/execctx/codehash_routing_test.go	Update derived codehash routing tests to new generation safety (cache-sourced record may seed mapping).
cmd/integration/commands/stages.go	Wire integration stage runner to canonical cache publication API.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 45 out of 45 changed files in this pull request and generated no new comments.

[Copilot]

Pull request overview

Copilot reviewed 45 out of 45 changed files in this pull request and generated no new comments.

Suppressed comments (2)

db/state/aggregator.go:1931

• When a commitment files publication triggers a BranchCache clear (BeginFilesPublication returns a non-nil backing-change), the AdaptivePinController should be reset as well; otherwise the controller will keep residency/miss state for pins that were just cleared and may make promotion/demotion decisions based on stale cache contents.

	// SharedDomains.Commit acquires cache publication in the same order.
	if domain := a.d[kv.CommitmentDomain]; domain != nil && domain.branchCache != nil {
		if commitmentVisible := visible.d[kv.CommitmentDomain]; commitmentVisible != nil {
			publication.branch = domain.branchCache.BeginFilesPublication(visibleFiles(commitmentVisible.files).EndTxNum())
		}
	}

db/state/aggregator.go:742

• closeDirtyFilesNoReopen resets the commitment BranchCache but does not reset the paired AdaptivePinController. Since AdaptivePinController keeps per-contract residency state, leaving it intact after a BranchCache reset can make the controller treat removed pins as still resident (see AdaptivePinController.Reset doc) and delay re-promotion/extension decisions.

This issue also appears on line 1926 of the same file.

	a.visibilityLoweringForbidden.Store(false)
	if cd := a.d[kv.CommitmentDomain]; cd != nil && cd.branchCache != nil {
		cd.branchCache.Reset()
	}