[longquanzheng]

Summary

Runs that existed before their partition offloaded history, started before the upgrade or while history offload was off, now migrate their local history to segments. The first command that leaves such a run running while its partition offloads opens its segment state at the run's first seq and event 1. Each later command of the run seals at most one segment, cut from history metadata at whole seqs, until a segment reaches the command itself. The run then seals like a run that started with offload on.

• A migration segment ends at the first seq where it holds two history records and reaches the run's segment target bytes, 10,000 superseded values, or 10,000 history rows, or at the command's own seq. Two records per segment let the migration gain on the run. Cuts are whole seqs, so one migration step costs at most about as much as the largest single command in the migrated range (for example, one legacy seq that superseded 12,000 values is sealed in one step).
• A segment holds the rows still stored in its seqs. Seqs that window pruning deleted before the migration are holes inside the reported pruned event range. Event ranges come from metadata; records older than last_assigned_event_id are decoded instead.
• RunRecord field 64, history_migration_started_seq, marks a migrating run; its open segment then tracks no bytes.
• Closed runs are not migrated, so retention purges their local history directly. A run that closes while migrating seals one more segment at its close and keeps the rest local until purge.
• Turning offload off ends a migration. Its unsealed rest joins the gap of commands applied while offload is off; those rows stay local and are pruned by the window rule.

Why

Refs #1325 (plan step B7, existing runs). Stacked on #1363 (B6, uploader); it will be retargeted to main once #1363 merges.

Before this change, a run without segment state in an offloading partition was fenced at uploaded seq 0. Window pruning stopped for it and none of its history reached object storage, so its local history grew without bound once offload was on. Migrating in bounded steps uploads that history without a spike.

Changes to B5 code: segment_seal.rs extracts write_sealed_segment and segment_target_bytes without changing behavior, and RunCursor::finish calls the migration-aware wrapper. B5's test runs_started_before_offload_keep_no_segment_state_and_keep_their_history is removed because B7 replaces that behavior. B5's simulation transition check (H9) now also accepts a migration start and a migration step, and requires progress on every command of a migrating run.

Compatibility

• RunRecord field 64 is additive. There are no new commands, tables, or key bytes.
• Apply results change only while the experimental history offload mode is on. Before server-zd 1.0 this needs no behavior version. Upgrade by stopping both standby and active, as for the earlier history offload changes.
• testdata/v1 is regenerated with its generator and its pin is updated. The workload appends legacy-running, legacy-closed, and legacy-migrating runs at its end.

Testing

• Plan test 1: format_compat a_recorded_running_run_migrates_one_segment_per_command_once_offload_is_on; apply a_running_legacy_run_migrates_one_segment_per_command_until_caught_up.
• Plan test 2: format_compat a_recorded_closed_run_is_not_migrated_and_its_purge_deletes_its_local_history; apply a_closed_legacy_run_is_not_migrated_and_its_purge_deletes_its_history.
• Plan test 3: format_compat time_travel_during_the_migration_of_a_recorded_run_is_correct; apply time_travel_during_a_migration_restores_its_target_and_the_migration_continues.
• Plan test 4: engine a_run_started_while_offload_was_off_uploads_all_its_history_once_offload_is_on (real uploader, in-memory object store); apply a_run_started_while_offload_was_off_migrates_once_it_is_on.
• Plan test 5: apply turning_offload_off_ends_a_migration_and_its_rest_becomes_a_gap and reenabling_offload_skips_the_gap_and_reads_report_its_pruned_events.
• Also: a close during a migration, records without last_assigned_event_id, commands larger than the target, a_legacy_seq_past_the_superseded_value_limit_migrates_in_one_step.
• Simulation: new invariants H11 (at most one sealed segment per run and command) and H12 (a migrating run tracks no open segment bytes); H9 requires a migration step on every command of a migrating run. The seed set requires migrations to start, complete, and be abandoned.
• Local: dexzd-apply 390 passed; dexzd-engine 482 passed (format_compat included); dexzd-simulation 34 passed; serve_process and dev_server history tests pass. cargo fmt --check, workspace clippy -D warnings, make rust-rules-check, make copyright-check, cd docs && npm run check pass.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!