Skip to content

refactor: implement runtime-owned mandatory maintenance - #930

Merged
jiangzhe merged 3 commits into
mainfrom
mandatory-maint
Aug 3, 2026
Merged

refactor: implement runtime-owned mandatory maintenance#930
jiangzhe merged 3 commits into
mainfrom
mandatory-maint

Conversation

@jiangzhe

@jiangzhe jiangzhe commented Aug 2, 2026

Copy link
Copy Markdown
Owner

Closes #928

Summary by CodeRabbit

  • Reliability

    • Improved checkpoint, freeze, redo cleanup, and index maintenance workflows.
    • Maintenance operations now complete safely even if the initiating request is cancelled or dropped.
    • Improved handling of concurrent checkpointing, publication conflicts, cancellations, and fatal storage errors.
  • Consistency

    • Catalog checkpointing is now performed through the session maintenance workflow, providing a unified and safer operation path.
  • Testing

    • Expanded coverage for checkpoint atomicity, recovery, cleanup completion, concurrency, and failure scenarios.

@jiangzhe jiangzhe self-assigned this Aug 2, 2026
@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: e92765e7-d431-4760-a575-25a8a46e27ac

📥 Commits

Reviewing files that changed from the base of the PR and between 1293182 and c33a982.

⛔ Files ignored due to path filters (2)
  • docs/rfcs/0026-engine-owned-mandatory-background-runtime.md is excluded by none and included by none
  • docs/tasks/000251-runtime-owned-mandatory-maintenance.md is excluded by none and included by none
📒 Files selected for processing (6)
  • doradb-storage/src/catalog/checkpoint.rs
  • doradb-storage/src/session.rs
  • doradb-storage/src/table/checkpoint_workflow.rs
  • doradb-storage/src/table/gc.rs
  • doradb-storage/src/table/persistence.rs
  • doradb-storage/src/trx/retention.rs

📝 Walkthrough

Walkthrough

The PR moves catalog, table, redo-retention, and MemIndex maintenance to prepared, lifetime-free operations owned by the mandatory runtime. It replaces borrowed leases with explicit scopes and engine-scoped maintenance test controls. Tests use session-level catalog checkpointing.

Changes

Runtime-owned maintenance

Layer / File(s) Summary
Admission scopes and redo operations
doradb-storage/src/catalog/checkpoint.rs, doradb-storage/src/latch/*, doradb-storage/src/trx/*
Catalog checkpoints and redo maintenance use explicit scopes, shared exclusive gates, prepared operations, accepted operations, and terminal cleanup.
Table workflow and lifecycle scopes
doradb-storage/src/table/checkpoint_workflow.rs, doradb-storage/src/table/lifecycle.rs, doradb-storage/src/table/access.rs
Table freeze, checkpoint, and root-mutation admission use lifetime-free attempts and owned scopes.
Prepared table checkpoint and cleanup execution
doradb-storage/src/table/persistence.rs, doradb-storage/src/table/gc.rs, doradb-storage/src/table/page_transition.rs
Table maintenance transfers ownership to mandatory execution and preserves publication, rollback, retry, poisoning, and completion behavior.
Session maintenance orchestration
doradb-storage/src/session.rs
Session methods prepare scopes, resolve live tables, submit mandatory operations, and use observer-based retry waits.
Engine-scoped maintenance controls
doradb-storage/src/engine.rs, doradb-storage/src/table/mod.rs, doradb-storage/src/table/persistence.rs
Test faults and asynchronous hooks move from thread-local or global state to cloneable, engine-scoped controllers.
Session checkpoint API migration
doradb-storage/src/catalog/*, doradb-storage/src/recovery/*, doradb-storage/src/table/recover.rs
Checkpoint tests use new_session().checkpoint_catalog() instead of removed direct checkpoint methods.

Estimated code review effort: 5 (Critical) | ~120 minutes

Possibly related PRs

  • jiangzhe/doradb#927: Refactors catalog admission from RAII leases to explicit acquire/release scopes.
  • jiangzhe/doradb#835: Introduces the table freeze/checkpoint workflow that this PR converts to lifetime-free attempts.
  • jiangzhe/doradb#915: Provides the session-operation coordinator used by the maintenance execution migration.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the primary change: moving mandatory maintenance ownership into the runtime.
Linked Issues check ✅ Passed The changes implement the listed maintenance migrations, prepared and accepted executions, owned scopes, runtime supervision, and related cancellation and cleanup behavior for issue #928.
Out of Scope Changes check ✅ Passed The changes support runtime-owned mandatory maintenance and its deterministic tests; no unrelated or out-of-scope code changes are evident.
Docstring Coverage ✅ Passed Docstring coverage is 84.70% which is sufficient. The required threshold is 80.00%.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch mandatory-maint

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@codacy-production

codacy-production Bot commented Aug 2, 2026

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 9 medium

Results:
9 new issues

Category Results
Complexity 9 medium

View in Codacy

🟢 Metrics 77 complexity · -12 duplication

Metric Results
Complexity 77
Duplication -12

View in Codacy

NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.

@codecov

codecov Bot commented Aug 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.39884% with 54 lines in your changes missing coverage. Please review.
✅ Project coverage is 93.35%. Comparing base (04265f8) to head (c33a982).
⚠️ Report is 2 commits behind head on main.

Files with missing lines Patch % Lines
doradb-storage/src/catalog/checkpoint.rs 84.76% 16 Missing ⚠️
doradb-storage/src/session.rs 96.78% 12 Missing ⚠️
doradb-storage/src/table/checkpoint_workflow.rs 91.26% 9 Missing ⚠️
doradb-storage/src/table/gc.rs 92.92% 8 Missing ⚠️
doradb-storage/src/table/lifecycle.rs 81.81% 8 Missing ⚠️
doradb-storage/src/table/persistence.rs 99.81% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #930      +/-   ##
==========================================
+ Coverage   93.33%   93.35%   +0.01%     
==========================================
  Files         148      149       +1     
  Lines      124538   125534     +996     
==========================================
+ Hits       116241   117191     +950     
- Misses       8297     8343      +46     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@greptile-apps

greptile-apps Bot commented Aug 2, 2026

Copy link
Copy Markdown

Greptile Summary

This PR moves mandatory maintenance operations from caller-owned futures to the engine’s supervised runtime so accepted work survives cancellation and observer loss.

  • Adds owned preparation and mandatory-execution scopes for table freeze/checkpoint, catalog checkpoint, redo truncation, and secondary-index cleanup.
  • Refactors checkpoint workflow, table lifecycle, redo-retention, and catalog publication coordination around the new ownership boundary.
  • Expands lifecycle, cancellation, concurrency, recovery, and failure-path coverage and documentation.

Confidence Score: 5/5

The PR appears safe to merge because no eligible new finding or known outstanding prior finding remains.

No blocking failure remains.

Important Files Changed

Filename Overview
doradb-storage/src/session.rs Introduces prepared and accepted maintenance scopes that transfer session-operation and logical-lock ownership to supervised execution.
doradb-storage/src/runtime/mandatory.rs Integrates the generalized accepted-maintenance execution contract with mandatory runtime admission and completion.
doradb-storage/src/table/persistence.rs Refactors table checkpoint execution and retry observation to operate with transferred workflow and root-mutation authority.
doradb-storage/src/table/checkpoint_workflow.rs Revises freeze and checkpoint attempt ownership while retaining restoration of canonical frozen batches on reversible exits.
doradb-storage/src/table/gc.rs Moves secondary MemIndex cleanup into accepted mandatory execution with owned table and transaction progress.
doradb-storage/src/catalog/checkpoint.rs Adds owned catalog-checkpoint resources and execution integration for mandatory maintenance.
doradb-storage/src/trx/retention.rs Coordinates catalog publication, durable redo-retention markers, and obsolete-file cleanup under transferred gates.
doradb-storage/src/trx/sys.rs Adds redo-retention scope ownership and progress tracking used by mandatory maintenance.
doradb-storage/src/table/lifecycle.rs Adds lifetime-independent checkpoint root-mutation authority for transfer across runtime admission.
doradb-storage/src/latch/gate.rs Introduces an asynchronous exclusive gate used to serialize maintenance domains.

Reviews (3): Last reviewed commit: "fix issues and resolve task" | Re-trigger Greptile

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
doradb-storage/src/table/checkpoint_workflow.rs (1)

320-394: 📐 Maintainability & Code Quality | 🟠 Major | 🏗️ Heavy lift

Deduplicate the two checkpoint admission paths and their drop logic.

Table::begin_checkpoint and TableCheckpointWorkflow::begin_checkpoint now contain the same admission state machine, and Drop for CheckpointAttempt and Drop for PreparedCheckpointAttempt contain the same restore logic. Only the receiver and the attempt type differ.

Both forms are needed: the retry-observation path in table/persistence.rs uses the borrowed attempt, and the prepared path uses the lifetime-free attempt. The risk is divergence. A later change to the Frozen versus Idle restore rules must be applied in two places, and a missed copy leaves the workflow in a wrong state without a compile error.

Extract the shared parts. Move the admission body into one private helper on TableCheckpointWorkflow that returns (CheckpointSource, Option<FrozenPageBatch>), and move the restore body into one private helper that takes &TableCheckpointWorkflow, CheckpointSource, and &mut Option<FrozenPageBatch>. Both entry points and both Drop implementations then call the helpers.

Also applies to: 609-693

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/table/checkpoint_workflow.rs` around lines 320 - 394,
Deduplicate checkpoint admission and restore logic by extracting the shared
state-machine body from both begin_checkpoint methods into a private
TableCheckpointWorkflow helper returning (CheckpointSource,
Option<FrozenPageBatch>). Extract the common Drop restoration logic into a
private helper accepting &TableCheckpointWorkflow, CheckpointSource, and &mut
Option<FrozenPageBatch>, then have both entry points and both Drop
implementations delegate to these helpers while preserving their borrowed and
lifetime-free attempt types.
doradb-storage/src/trx/retention.rs (1)

368-384: 📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Update the doc comment to the scope model.

The doc still describes a "catalog checkpoint lease" and a "redo-retention lease" acquired in this method. This PR replaces both with CatalogCheckpointScope and RedoRetentionScope, and the caller acquires them before preparation. The method now only releases catalog authority through release_catalog. Rewrite the ordering paragraph so it states the caller-held scopes and the release point.

📝 Proposed doc update
-    /// Lock ordering matches catalog checkpoint: acquire the catalog checkpoint
-    /// lease before the redo-retention lease. The catalog lease protects the
-    /// `catalog.mtb` root fork used to publish `first_redo_log_seq`, while the
-    /// redo-retention lease protects the retained redo suffix, catalog-safe
-    /// progress cache, and cleanup below the marker. They are separate because
-    /// the marker is catalog bootstrap metadata, but unlink races are about the
-    /// redo file family rather than catalog metadata shape.
+    /// The caller acquires `CatalogCheckpointScope` before `RedoRetentionScope`
+    /// and transfers both into the accepted operation. The catalog scope
+    /// protects the `catalog.mtb` root fork used to publish
+    /// `first_redo_log_seq`. The redo-retention scope protects the retained
+    /// redo suffix, the catalog-safe progress cache, and cleanup below the
+    /// marker. This method releases only catalog authority, through
+    /// `release_catalog`, before filesystem cleanup. The scopes are separate
+    /// because the marker is catalog bootstrap metadata, but unlink races are
+    /// about the redo file family rather than catalog metadata shape.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/trx/retention.rs` around lines 368 - 384, Update the doc
comment for truncate_redo_log_prepared to describe caller-held
CatalogCheckpointScope and RedoRetentionScope rather than leases acquired in the
method, preserving their ordering and responsibilities. State that the method
only releases catalog authority through the release_catalog callback, and remove
claims that it acquires either scope.
🧹 Nitpick comments (7)
doradb-storage/src/table/checkpoint_workflow.rs (1)

771-783: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add coverage for PreparedCheckpointAttempt drop restore.

This test exercises the borrowed TableCheckpointWorkflow::begin_checkpoint and Drop for CheckpointAttempt. The new Table::begin_checkpoint and Drop for PreparedCheckpointAttempt have no equivalent test in this file. table/persistence.rs adds test_prepared_freeze_attempt_drop_restores_idle for the freeze path only.

Add a test that admits a checkpoint through Table::begin_checkpoint from a Frozen state, drops the attempt, and asserts the workflow returns to Frozen with the batch intact. That path is where the duplicated restore logic can diverge.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/table/checkpoint_workflow.rs` around lines 771 - 783, Add
a test alongside test_reversible_checkpoint_attempt_restores_admitted_state
covering Table::begin_checkpoint and PreparedCheckpointAttempt drop. Transition
a Table to Frozen with an admitted batch, begin and drop the checkpoint attempt,
then assert the workflow is Frozen and the original batch remains intact.
doradb-storage/src/table/persistence.rs (2)

2305-2320: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add a guard for the secondary-sidecar failure flag.

set_test_force_secondary_sidecar_error sets an engine-scoped flag with no RAII reset, while the two neighboring flags provide ForcePostPublishCheckpointErrorGuard and ForceCheckpointCommitErrorGuard. test_secondary_sidecar_failure_keeps_checkpoint_root_atomic therefore hand-rolls ResetSidecarHook. Because the flag now lives on the engine instead of a thread-local, a test that forgets the reset leaves the failure injected for every later operation on that engine.

Add a ForceSecondarySidecarErrorGuard next to the other two guards and use it in the test.

♻️ Proposed guard
+        pub(crate) struct ForceSecondarySidecarErrorGuard {
+            test: MaintenanceTestController,
+        }
+
+        impl ForceSecondarySidecarErrorGuard {
+            pub(crate) fn new(engine: &Engine) -> Self {
+                let test = engine.inner().maintenance_test.clone();
+                test.set_force_secondary_sidecar_error(true);
+                Self { test }
+            }
+        }
+
+        impl Drop for ForceSecondarySidecarErrorGuard {
+            fn drop(&mut self) {
+                self.test.set_force_secondary_sidecar_error(false);
+            }
+        }

Then in the test:

-        struct ResetSidecarHook(MaintenanceTestController);
-
-        impl Drop for ResetSidecarHook {
-            fn drop(&mut self) {
-                self.0.set_force_secondary_sidecar_error(false);
-            }
-        }
-
...
-            set_test_force_secondary_sidecar_error(&engine, true);
-            let _reset = ResetSidecarHook(engine.inner().maintenance_test.clone());
+            let _reset = ForceSecondarySidecarErrorGuard::new(&engine);

Also applies to: 3316-3357

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/table/persistence.rs` around lines 2305 - 2320, Add a
RAII-style ForceSecondarySidecarErrorGuard alongside
ForcePostPublishCheckpointErrorGuard and ForceCheckpointCommitErrorGuard,
ensuring it enables the engine-scoped secondary-sidecar failure flag on creation
and resets it on drop. Update
test_secondary_sidecar_failure_keeps_checkpoint_root_atomic to use this guard
instead of the hand-rolled ResetSidecarHook, while preserving the existing
failure-injection behavior.

4055-4102: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Make the observer drop explicit.

The test name states that the observer is dropped, but the freeze future is dropped implicitly when the block at lines 4071-4099 ends. A reader must infer the drop point from the brace. The other dropped-observer tests in this file call drop(checkpoint) explicitly.

Bind the future outside the block and call drop before release_tx.send_async, so the acceptance-boundary behavior under test is visible at the assertion site.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/table/persistence.rs` around lines 4055 - 4102, Make the
freeze future’s cancellation explicit in
test_dropped_freeze_observer_does_not_cancel_loading: bind the freeze future
outside the scoped block, retain the existing polling and competing-session
assertions, then call drop on the freeze future immediately before
release_tx.send_async. Ensure the observer is visibly dropped at the acceptance
boundary rather than only by scope exit.
doradb-storage/src/session.rs (3)

492-574: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoff

Consider sharing the accepted-scope state machine with AcceptedDdlScope.

MaintenanceFinishState and AcceptedMaintenanceScope duplicate DdlFinishState and AcceptedDdlScope at lines 300-378. The only difference is catalog_write_authority. The finish, handle_panic, and mark_terminal_ready bodies are identical, so a later fix to the terminal-edge ordering must be applied twice.

A generic carrier over the retained lock type, or a shared inner struct that both scopes embed, would keep one implementation of the terminal-state rules. This is not required before merge.

Also note lines 505-516 and 564-574: AcceptedMaintenanceScope defines inherent engine() and pool_guards() that shadow the identical SessionRuntimeAccess members. Removing the inherent copies keeps one resolution path.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/session.rs` around lines 492 - 574, Share the finish-state
and terminal-transition implementation between AcceptedDdlScope and
AcceptedMaintenanceScope by introducing a generic carrier or shared inner state
over their differing retained lock types, so finish, handle_panic, and
mark_terminal_ready have one implementation. Remove the redundant inherent
engine() and pool_guards() methods from AcceptedMaintenanceScope and rely on
SessionRuntimeAccess for those members.

615-654: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚖️ Poor tradeoff

Optional: fold the remaining ScopedTableRuntimeAccess into the prepared scope.

After this PR, ScopedTableRuntimeAccess serves only total_row_pages at line 1321. It acquires the same metadata S plus data IS set as PreparedMaintenanceLocks::acquire_table, but through acquire_table_locks instead of OwnerLockState. The two paths currently agree on resource order. Keeping one acquisition path would remove the risk that a future order change is applied to only one of them.

total_row_pages does not submit a mandatory operation, so this is not required now.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/session.rs` around lines 615 - 654, Optionally remove the
standalone ScopedTableRuntimeAccess acquisition path and fold total_row_pages
into the existing PreparedMaintenanceLocks::acquire_table flow. Reuse
OwnerLockState-based lock acquisition and the prepared scope’s live-table
resolution, preserving metadata S plus data IS ordering and total_row_pages
behavior; update only the affected helper and call sites.

466-478: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Delegate resolve_user_table to the pinned operation.

PreparedMaintenanceScope::resolve_user_table repeats the body of SessionOperationPin::resolve_user_table at lines 1561-1572. The two copies can diverge when catalog validation or cache behavior changes.

♻️ Proposed delegation
     pub(crate) async fn resolve_user_table(
         &self,
         table_id: TableID,
     ) -> OperationResult<Arc<Table>> {
-        let table = self
-            .operation
-            .engine
-            .catalog()
-            .validate_user_table_live(table_id)
-            .await?;
-        self.operation.state.cache_user_table(&table);
-        Ok(table)
+        self.operation.resolve_user_table(table_id).await
     }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/session.rs` around lines 466 - 478, Update
PreparedMaintenanceScope::resolve_user_table to delegate directly to the pinned
SessionOperationPin::resolve_user_table implementation instead of repeating
catalog validation and cache logic; preserve the existing table_id input and
OperationResult<Arc<Table>> behavior.
doradb-storage/src/table/mod.rs (1)

1509-1575: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Document that FnMut hooks run while the controller lock is held.

run_frozen_page_scan_hook, run_frozen_page_row_scan_hook, and run_optimistic_page_plan_comparison_hook invoke the closure while holding the parking_lot::Mutex. The previous thread-local hooks could not be reached from another thread. This controller is engine-scoped and shared across mandatory-runtime threads, so a hook that blocks now stalls every other thread that reaches the same hook, and a hook that re-enters the same scan path self-deadlocks.

The existing hook in doradb-storage/src/table/page_transition.rs at line 1001 sends on a flume::bounded(1) channel and is safe only because the channel has spare capacity. Add a doc comment on these three methods that states the closure must not block and must not re-enter the scan path.

The one-shot async runners already take the hook out of the lock before awaiting, so they are unaffected.

📝 Proposed documentation
+        /// Runs the frozen-page scan hook.
+        ///
+        /// The hook runs while the shared controller lock is held. The hook
+        /// must not block and must not re-enter frozen-page scanning.
         pub(crate) fn run_frozen_page_scan_hook(&self, page_id: PageID) {
             if let Some(hook) = self.state.frozen_page_scan_hook.lock().as_mut() {
                 hook(page_id);
             }
         }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/table/mod.rs` around lines 1509 - 1575, Add doc comments
to run_frozen_page_scan_hook, run_frozen_page_row_scan_hook, and
run_optimistic_page_plan_comparison_hook stating that each FnMut closure
executes while the controller Mutex is held, must not block, and must not
re-enter the scan path. Leave the one-shot async runners unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@doradb-storage/src/catalog/checkpoint.rs`:
- Around line 354-442: Replace the duplicated prepared/accepted carriers with
one generic shared carrier that owns AcceptedMaintenanceScope, each operation’s
resource tuple, and panic-label data, implementing finish and handle_panic once.
Update doradb-storage/src/catalog/checkpoint.rs:354-442,
doradb-storage/src/trx/retention.rs:120-215 and 217-313,
doradb-storage/src/table/persistence.rs:127-238 and 240-349, and
doradb-storage/src/table/gc.rs:183-291 so each operation supplies only its
execute body and resources; preserve catalog/redo release order, release_catalog
callbacks, attempt-before-root-mutation ordering, and MemIndexCleanupPhase panic
attachment.

In `@doradb-storage/src/table/gc.rs`:
- Around line 301-375: Bound the retry loop in execute_inner by tracking
attempts and, after a small fixed maximum, return a RuntimeError::TableAccess
report containing the table ID and attempt count. Increment the counter for each
iteration, preserve the existing rollback and retry behavior below the limit,
and keep successful cleanup unchanged.

---

Outside diff comments:
In `@doradb-storage/src/table/checkpoint_workflow.rs`:
- Around line 320-394: Deduplicate checkpoint admission and restore logic by
extracting the shared state-machine body from both begin_checkpoint methods into
a private TableCheckpointWorkflow helper returning (CheckpointSource,
Option<FrozenPageBatch>). Extract the common Drop restoration logic into a
private helper accepting &TableCheckpointWorkflow, CheckpointSource, and &mut
Option<FrozenPageBatch>, then have both entry points and both Drop
implementations delegate to these helpers while preserving their borrowed and
lifetime-free attempt types.

In `@doradb-storage/src/trx/retention.rs`:
- Around line 368-384: Update the doc comment for truncate_redo_log_prepared to
describe caller-held CatalogCheckpointScope and RedoRetentionScope rather than
leases acquired in the method, preserving their ordering and responsibilities.
State that the method only releases catalog authority through the
release_catalog callback, and remove claims that it acquires either scope.

---

Nitpick comments:
In `@doradb-storage/src/session.rs`:
- Around line 492-574: Share the finish-state and terminal-transition
implementation between AcceptedDdlScope and AcceptedMaintenanceScope by
introducing a generic carrier or shared inner state over their differing
retained lock types, so finish, handle_panic, and mark_terminal_ready have one
implementation. Remove the redundant inherent engine() and pool_guards() methods
from AcceptedMaintenanceScope and rely on SessionRuntimeAccess for those
members.
- Around line 615-654: Optionally remove the standalone ScopedTableRuntimeAccess
acquisition path and fold total_row_pages into the existing
PreparedMaintenanceLocks::acquire_table flow. Reuse OwnerLockState-based lock
acquisition and the prepared scope’s live-table resolution, preserving metadata
S plus data IS ordering and total_row_pages behavior; update only the affected
helper and call sites.
- Around line 466-478: Update PreparedMaintenanceScope::resolve_user_table to
delegate directly to the pinned SessionOperationPin::resolve_user_table
implementation instead of repeating catalog validation and cache logic; preserve
the existing table_id input and OperationResult<Arc<Table>> behavior.

In `@doradb-storage/src/table/checkpoint_workflow.rs`:
- Around line 771-783: Add a test alongside
test_reversible_checkpoint_attempt_restores_admitted_state covering
Table::begin_checkpoint and PreparedCheckpointAttempt drop. Transition a Table
to Frozen with an admitted batch, begin and drop the checkpoint attempt, then
assert the workflow is Frozen and the original batch remains intact.

In `@doradb-storage/src/table/mod.rs`:
- Around line 1509-1575: Add doc comments to run_frozen_page_scan_hook,
run_frozen_page_row_scan_hook, and run_optimistic_page_plan_comparison_hook
stating that each FnMut closure executes while the controller Mutex is held,
must not block, and must not re-enter the scan path. Leave the one-shot async
runners unchanged.

In `@doradb-storage/src/table/persistence.rs`:
- Around line 2305-2320: Add a RAII-style ForceSecondarySidecarErrorGuard
alongside ForcePostPublishCheckpointErrorGuard and
ForceCheckpointCommitErrorGuard, ensuring it enables the engine-scoped
secondary-sidecar failure flag on creation and resets it on drop. Update
test_secondary_sidecar_failure_keeps_checkpoint_root_atomic to use this guard
instead of the hand-rolled ResetSidecarHook, while preserving the existing
failure-injection behavior.
- Around line 4055-4102: Make the freeze future’s cancellation explicit in
test_dropped_freeze_observer_does_not_cancel_loading: bind the freeze future
outside the scoped block, retain the existing polling and competing-session
assertions, then call drop on the freeze future immediately before
release_tx.send_async. Ensure the observer is visibly dropped at the acceptance
boundary rather than only by scope exit.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 52193552-0ae5-4012-a466-dc5e19c1f077

📥 Commits

Reviewing files that changed from the base of the PR and between 947ff81 and 58bad7c.

⛔ Files ignored due to path filters (9)
  • docs/architecture.md is excluded by none and included by none
  • docs/checkpoint.md is excluded by none and included by none
  • docs/engine-component-lifetime.md is excluded by none and included by none
  • docs/lock-system.md is excluded by none and included by none
  • docs/public-error-audit.csv is excluded by !**/*.csv and included by none
  • docs/tasks/000251-runtime-owned-mandatory-maintenance.md is excluded by none and included by none
  • docs/tasks/next-id is excluded by none and included by none
  • docs/transaction-system.md is excluded by none and included by none
  • docs/unsafe-usage-baseline.md is excluded by none and included by none
📒 Files selected for processing (22)
  • doradb-storage/src/catalog/checkpoint.rs
  • doradb-storage/src/catalog/index.rs
  • doradb-storage/src/catalog/mod.rs
  • doradb-storage/src/catalog/storage/mod.rs
  • doradb-storage/src/catalog/table.rs
  • doradb-storage/src/engine.rs
  • doradb-storage/src/latch/gate.rs
  • doradb-storage/src/latch/mod.rs
  • doradb-storage/src/recovery/mod.rs
  • doradb-storage/src/runtime/mandatory.rs
  • doradb-storage/src/session.rs
  • doradb-storage/src/table/access.rs
  • doradb-storage/src/table/checkpoint_workflow.rs
  • doradb-storage/src/table/gc.rs
  • doradb-storage/src/table/lifecycle.rs
  • doradb-storage/src/table/mod.rs
  • doradb-storage/src/table/page_transition.rs
  • doradb-storage/src/table/persistence.rs
  • doradb-storage/src/table/recover.rs
  • doradb-storage/src/trx/mod.rs
  • doradb-storage/src/trx/retention.rs
  • doradb-storage/src/trx/sys.rs
💤 Files with no reviewable changes (1)
  • doradb-storage/src/runtime/mandatory.rs

Comment thread doradb-storage/src/catalog/checkpoint.rs Outdated
Comment thread doradb-storage/src/table/gc.rs Outdated

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
doradb-storage/src/session.rs (1)

1174-1183: 🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

Settle the maintenance operation when preparing without accepted scope.

The return paths at poisoner.ensure_healthy and the prepare_*_operation failures drop PreparedMaintenanceScope, but they never call the mandatory finish/fail_retained flow, so the active operation stays retained. These global-scope methods can also leave catalog and redo retention guards held if the scope is dropped before work starts.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@doradb-storage/src/session.rs` around lines 1174 - 1183, Update the global
maintenance preparation flow around PreparedMaintenanceScope,
poisoner.ensure_healthy, and prepare_catalog_checkpoint_operation so every early
failure settles the operation through the mandatory finish/fail_retained path
before returning. Ensure catalog_scope and redo_scope retention guards are
explicitly released or settled when preparation fails or the scope is dropped
before work starts.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@doradb-storage/src/session.rs`:
- Around line 1174-1183: Update the global maintenance preparation flow around
PreparedMaintenanceScope, poisoner.ensure_healthy, and
prepare_catalog_checkpoint_operation so every early failure settles the
operation through the mandatory finish/fail_retained path before returning.
Ensure catalog_scope and redo_scope retention guards are explicitly released or
settled when preparation fails or the scope is dropped before work starts.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 0959bfe8-031c-4da4-a659-782670b33cf9

📥 Commits

Reviewing files that changed from the base of the PR and between 58bad7c and 1293182.

📒 Files selected for processing (10)
  • doradb-storage/src/catalog/checkpoint.rs
  • doradb-storage/src/catalog/mod.rs
  • doradb-storage/src/catalog/table.rs
  • doradb-storage/src/session.rs
  • doradb-storage/src/table/gc.rs
  • doradb-storage/src/table/mod.rs
  • doradb-storage/src/table/persistence.rs
  • doradb-storage/src/table/recover.rs
  • doradb-storage/src/trx/mod.rs
  • doradb-storage/src/trx/retention.rs
🚧 Files skipped from review as they are similar to previous changes (6)
  • doradb-storage/src/trx/mod.rs
  • doradb-storage/src/catalog/table.rs
  • doradb-storage/src/table/recover.rs
  • doradb-storage/src/table/mod.rs
  • doradb-storage/src/table/gc.rs
  • doradb-storage/src/table/persistence.rs

@jiangzhe
jiangzhe merged commit aa71c52 into main Aug 3, 2026
13 of 16 checks passed
@jiangzhe
jiangzhe deleted the mandatory-maint branch August 3, 2026 04:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Task: Runtime-Owned Mandatory Maintenance

1 participant