Repository navigation
server-zd: read history through a HistorySource and select time travel targets from metadata - #1348
Merged
Merged
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
longquanzheng
force-pushed
the
history-offload/b4-history-source
branch
from
November 7, 2026 04:48
2995a3f to
df27ebd
Compare
…argets from metadata History offload will move History records and checkpoints out of the local store, so time travel and the history read model must stop reading them from the transaction directly. - dexzd-apply defines HistorySource: a synchronous source that streams a run's History records by seq range as an iterator and returns a checkpoint by (attempt, seq). LocalHistorySource reads the History rows of a FlowTransaction in bounded batches, as history_records and the read model's scans did, and checks each key against its record's seq. - rebuild_attempt_state, collect_external_inputs, resolve_target, and the GetHistoryEvents read model read records and checkpoints only through the source. resolve_target and collect_external_inputs take run rows and the run record instead of a RunCursor, so the engine can call them. get_history_events keeps its signature; get_history_events_from_source takes any source. - resolve_target reads candidates from HistoryRowMetadata fields 7-9 and stops at the first match, with the same last_event_id and FlowClosed message rules as before. A run whose genesis metadata has no largest event ID predates those fields, because FlowStart always records event 1, so it falls back to reading records through the source. - Tests: a simulation test compares metadata, record, and the previous eager resolution over random histories with forks and window pruning for every target type, checks that metadata resolution reads no History records, checks the fallback with stripped metadata, and compares history pages read through an in-memory source with local reads. An apply test checks TimeTravel outcomes over older metadata.
…ation The page comparison passed the store that still holds every History row to get_history_events_from_source, so a direct local read or an accidental fallback would also pass. It now reads through a reader that fails every History column access, and checks that a local read through that reader fails. Pages are compared both without a response byte limit and with a small one, so pages that end early or fail on one large event must also match.
longquanzheng
force-pushed
the
history-offload/b4-history-source
branch
from
November 7, 2026 05:28
a7ebb60 to
6616e1d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
HistorySourceto dexzd-apply. It is a synchronous source that streams a run's History records by seq range and returns checkpoints by (attempt, seq).LocalHistorySourcereads the transaction's History rows in bounded batches, as before, and checks each key seq against its record's seq.rebuild_attempt_state,collect_external_inputs,resolve_target, and the GetHistoryEvents read model read History only through the source.resolve_targetandcollect_external_inputstake run rows and the run record instead of aRunCursor.get_history_eventskeeps its signature, and the newget_history_events_from_sourceaccepts any source.resolve_targetselects from HistoryRowMetadata fields 7–9 and stops at the first match, with the samelast_event_idand FlowClosed message rules as before. A run whose genesis metadata has no largest event ID predates those fields (FlowStart always records event 1), so it falls back to streaming records through the source.Why
History offload (Refs #1325, plan step B4) moves History records and checkpoints to object storage. Time travel and GetHistoryEvents must read them through one abstraction, so that later steps only add a merged object-storage-plus-local-tail source and the same code runs in apply and in the engine. Target selection must also stop reading the whole window's records. #1342 added the metadata fields this PR reads; this PR is stacked on it and will be retargeted to main once #1342 merges.
Testing
cargo test -p dexzd-apply(347 passed),-p dexzd-engine(456 passed, including format_compat),-p dexzd-simulation(33 passed). Existing suites are unchanged.dexzd-simulation/tests/time_travel_targets.rs(16 seeds × 250 commands by default): after every command, metadata, record, and the previous eager resolution agree on every run attempt for all four target types and invalid requests; metadata resolution succeeds with a source that fails every read; stripped metadata falls back with identical results (forced metadata on stripped rows misresolves 5443 requests, so the fallback is exercised); history pages read through an in-memory source match local reads.cargo fmt --check, clippy-D warnings --all-targets,cargo check --workspace --all-targets,make rust-rules-check,make copyright-check.