diff --git a/docs/architecture.md b/docs/architecture.md index 9d436f3e..4bb931c1 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -163,9 +163,16 @@ Engine lifecycle admission closes session operation and inspection registration against shutdown. After admission drops, stable session operation entries account effectful foreground work, per-session observer counts account standalone diagnostics and progress waits, and mandatory permits account -accepted caller or internal cleanup work. The crate-private `EngineRef` remains -an `Arc` access wrapper for memory reachability; cloning it does -not create a separate shutdown blocker. +accepted caller or internal cleanup work. Public session and transaction +handles retain weak reachability to their exact `SessionState`. Successful +admission returns a short-lived admitted session wrapper that alone exposes the +normal weak-state upgrade. Consuming that wrapper produces an admitted +`SessionRuntime`, retaining the same admission until the stable operation or +observer proof is registered. Operation and transaction identity then resolve +directly on the pinned state without a session-registry lookup. The state +reaches immutable component capabilities through `EngineCore`, whose only +registry back-reference is weak and used for cold pointer-exact removal after a +session becomes closed and idle. ## Logging, Checkpoint and Recovery diff --git a/docs/backlogs/000175-scalable-shared-resource-lifetime-management.md b/docs/backlogs/000175-scalable-shared-resource-lifetime-management.md index dec55c97..9257e383 100644 --- a/docs/backlogs/000175-scalable-shared-resource-lifetime-management.md +++ b/docs/backlogs/000175-scalable-shared-resource-lifetime-management.md @@ -23,19 +23,55 @@ Phase 2 accepted this fixed boundary cost as explicit performance debt because its cancellation-ownership prerequisite is complete; this backlog owns the performance correction rather than treating the original budget as passed. +Task 000255 removed the engine-wide weak upgrade, session-registry operation +lookup, and attachment `PoolGuards` clone from session-coordinated statement +checkout. Its paired release measurements against +`2098cbb70316d383881aa3c05ba6ef56db408cc3` reduced median `stmt-noop` latency +from 73.524 ns to 47.865 ns at 1 thread/1 session and from 83.893 ns to +76.425 ns at 4 threads/16 sessions. Median `trx-noop` latency likewise fell +from 301.035 ns to 223.491 ns at 1/1 and from 264.610 ns to 220.616 ns at +4/16. + +The same matrix exposed a separate contended `index-stream` result. At +4 threads/16 sessions, the unique-index median increased from 76,799 ns to +106,144 ns per stream and the non-unique median increased from 82,875 ns to +105,518 ns. The 1/1 rows were near-neutral by comparison: unique increased +from 233,689 ns to 240,751 ns and non-unique decreased from 240,770 ns to +238,612 ns. Independent repeated 4/16 blocks reproduced the unfavorable +result. + +Paired `cargo flamegraph` profiles localized the extra candidate CPU to +existing buffer and row-page reference-count operations rather than the new +session runtime path. In warmed unique-index 4/16 profiles, relaxed and release +`Arc` atomic helpers accounted for about 29.75% of candidate samples versus +about 4.65% of baseline samples, primarily below row-page lookup, fixed-buffer +page lookup, and page-frame release. Candidate `SessionRuntime` and attachment +pool-guard access accounted for about 0.07%; no weak-engine upgrade, +session-registry lookup, or attachment guard-bundle clone remained in the +candidate stack. This evidence identifies the contended domain but does not +yet prove why the ownership-path speedup changes page-frame contention. + ## Deferred From (Optional) -docs/tasks/000247-statement-public-transaction-cancellation-ownership.md; docs/rfcs/0025-session-coordinated-cancellation-cleanup-ownership.md Phase 2 +docs/tasks/000247-statement-public-transaction-cancellation-ownership.md; docs/rfcs/0025-session-coordinated-cancellation-cleanup-ownership.md Phase 2; docs/tasks/000255-session-local-runtime-reachability.md ## Deferral Context (Optional) -- Defer Reason: Task 000247 is scoped to statement cancellation ownership and its bounded performance work. Redesigning lifetime and destruction policy across the engine, pools, transaction system, and other component resources materially broadens both architecture and shutdown proof obligations, so it should be planned and reviewed independently rather than folded into Phase 2. -- Findings: The session coordinator is already an authoritative teardown blocker for active public/private transactions and foreground operations, making the custom counted `EngineRef` pin redundant for much of that hot path. It is not yet authoritative for every runtime user: detached observer pins do not occupy the active operation slot, and standalone/internal strong pins rely on `runtime_refs` for efficient shutdown notification. `Arc::strong_count` remains the final ownership backstop, but it has no drop notification, explaining why the separate counter exists. Removing local `TrxInner` copy work exposed contention on shared lifecycle cache lines rather than adding new lifecycle operations. +- Defer Reason: Task 000247 is scoped to statement cancellation ownership and its bounded performance work. Redesigning lifetime and destruction policy across the engine, pools, transaction system, and other component resources materially broadens both architecture and shutdown proof obligations, so it should be planned and reviewed independently rather than folded into Phase 2. Task 000255 is scoped to session-local runtime reachability; changing buffer-frame ownership or index-stream scheduling to address the newly measured contention would cross that boundary without a proven cause. +- Findings: The session coordinator is already an authoritative teardown blocker for active public/private transactions and foreground operations, making the custom counted `EngineRef` pin redundant for much of that hot path. It is not yet authoritative for every runtime user: detached observer pins do not occupy the active operation slot, and standalone/internal strong pins rely on `runtime_refs` for efficient shutdown notification. `Arc::strong_count` remains the final ownership backstop, but it has no drop notification, explaining why the separate counter exists. Removing local `TrxInner` copy work exposed contention on shared lifecycle cache lines rather than adding new lifecycle operations. Task 000255 confirmed that session-local weak reachability materially improves statement and transaction no-op paths. Its contended index-stream profiles show increased time in existing page-frame `Arc` increments and decrements, while the new session runtime access is negligible; the relationship between faster statement boundaries and buffer-page contention remains unproven. - Direction Hint: Start with the narrow performance result: make session-coordinated transaction and foreground-operation access use the session/component lifecycle proof instead of globally counted runtime pins, while retaining admission for the operation-start versus shutdown race. Explicitly account for detached observers, terminal-publication gaps, stale cleanup jobs, and worker-owned work before narrowing or removing `runtime_refs`. Then evaluate the general resource-lifetime policy. Compare sharded counters with centralized arena/owner destruction rather than assuming one universal mechanism. Prefer centralized destruction when resource lifetime is already bounded by an engine/component owner and individual early reclamation is unnecessary; prefer sharding only where independent lifetime and thread mobility still require counting. Avoid weakening memory ordering or deleting counters without a replacement shutdown and destruction proof. + Reproduce the task 000255 index-stream profile independently before changing + buffer or index code. Separate page-frame atomic operation count from + per-operation contention latency, and test whether statement-boundary + synchronization, shared root traversal, range overlap, or object/cache-line + placement explains the 4/16-only result. Do not optimize index-stream by + adding duplicate runtime capabilities or unsafe cached pointers unless a + profile-backed design proves that session runtime access is causal. + ## Scope Hint Inventory high-frequency shared lifetime counters and guard cloning across engine access, buffer pools, transaction-system access, catalog/table runtime access, and related long-lived resources. Separate session-coordinated owners from detached observers and worker-owned pins. Remove or amortize global counter operations where an authoritative session/component owner already proves liveness. Evaluate sharded reference counters and centralized arena/owner-managed destruction against thread mobility, shutdown ordering, reclamation latency, memory safety, and measured contention. Use an RFC if the chosen direction changes ownership architecture across multiple subsystems. diff --git a/docs/engine-component-lifetime.md b/docs/engine-component-lifetime.md index c76530f4..3668d514 100644 --- a/docs/engine-component-lifetime.md +++ b/docs/engine-component-lifetime.md @@ -7,16 +7,22 @@ component-registry migration work. ## Terminology - `Engine`: public owner of top-level teardown state and session creation. -- `EngineInner`: crate-private shared runtime state held behind the engine - owner and internal shared handles. -- `EngineRef`: crate-private cloneable `Arc` access wrapper. It - provides memory reachability and component access but is not itself a - shutdown blocker. +- `EngineInner`: owner-facing coordination shell containing `EngineCore`, the + strong session registry, the lifecycle gate, and the session-id source. +- `EngineCore`: immutable component-capability set retained by registered + session state. It has only a weak back-reference to the session registry. +- `SessionRuntime`: typed strong reference to one exact `SessionState`. +- `WeakSessionRef`: weak reference to one exact `SessionState` plus that + session's limited lifecycle-admission façade. +- `AdmittedSessionRef`: short-lived pairing of that exact weak state with the + admission acquired through its session façade. +- `AdmittedSessionRuntime`: the result of consuming an admitted weak reference + and upgrading it while retaining the same admission. - Public session and transaction handles: weak, non-cloneable capabilities that - identify engine-local state and acquire admitted internal access only for one + identify exact session-local state and acquire admitted internal access for one operation or terminal path. - `SessionOperationEntry`: one registry-owned stable operation record keyed by - `(SessionID, OperationID)`; it contains no `EngineRef`, + `(SessionID, OperationID)`; it contains no engine-wide reference, `SessionObserverPin`, or whole operation future. - `SessionObserverPin`: non-cloneable standalone observer authority accounted by its session lifecycle without consuming the effectful operation slot. @@ -31,8 +37,12 @@ The runtime uses an explicit owner/runtime split: - `Engine` owns: - `inner: Arc` - `components: ComponentRegistry` -- `EngineInner` owns only crate-private shared runtime handles and the - lifecycle gate: +- `EngineInner` owns: + - `core: Arc` + - `session_registry: Arc` + - `lifecycle: Arc` + - the engine-local session-id source +- `EngineCore` owns the shared runtime capabilities: - engine poisoner - mandatory runtime - catalog @@ -41,7 +51,12 @@ The runtime uses an explicit owner/runtime split: - fixed and evictable buffer pools - table-file subsystem - readonly buffer pool - - shutdown admission state + - a weak session-registry back-reference used only for cold exact removal + +The registry owns each `Arc`. Each state retains `Arc` +and one `Arc` into the lifecycle gate. `EngineCore` does not +retain `EngineInner`, the lifecycle gate, or a strong registry reference, so +the graph has no strong cycle. `ComponentRegistry` is intentionally not part of `EngineInner`. The registry is needed only for explicit reverse-order shutdown and final owner drop. Keeping @@ -105,7 +120,7 @@ dependency. `MandatoryRuntime` is registered immediately after the poisoner. Catalog, transaction, recovery, and future operation adapters can therefore retain its -direct `QuiescentGuard` without owning `EngineRef` or another runtime `Arc`. +direct `QuiescentGuard` without owning the engine owner shell. Its build shelves only the runtime guard and configured runner count. The later `MandatoryRuntimeWorkers` build starts the fixed runners and registers their join-handle owner at the required shutdown position. @@ -370,37 +385,42 @@ The lazy traversal may hold one DashMap shard read guard during the short either inner mutex is held, so there is no reverse lock edge. The iterator is dropped before cleanup submission, event waiting, notification, or removal. -The registry owns `Arc`, and an active slot owns -`Arc`. Neither object owns a strong engine runtime -handle. `EngineRef` exists only in scoped foreground authorities, transaction -or observer authorities, transaction attachments, claims, and submitted -cleanup jobs, preventing a registry-to-engine strong reference cycle. Engine -admission closes every new operation or observer registration against shutdown; -session entries and observer counts then become the durable shutdown proof -after admission drops. Mandatory permits provide the corresponding proof for -accepted caller and internal cleanup work. +The registry owns `Arc`, each state owns `Arc`, and an +active slot owns `Arc`. Public `Session` and +`Transaction` handles own only `WeakSessionRef`. Operation authorities, +transaction attachments, claims, and cleanup jobs retain `SessionRuntime`, so +they reach components through the already-pinned exact state without recovering +`EngineInner` or looking up the registry. Engine admission closes every new +operation or observer registration against shutdown; session entries and +observer counts then become the durable shutdown proof after admission drops. +Mandatory permits provide the corresponding proof for accepted caller and +internal cleanup work. The owned-handle inventory follows those authorities: -- `SessionObserverPin` pairs its `EngineRef` with one counted session observer. +- `SessionObserverPin` pairs `SessionRuntime` with one counted session observer. - `SessionOperationPin`, `TrxAttachment`, transaction checkout and completion - claims, and DDL or maintenance progress all remain paired with their exact - stable `SessionOperationEntry`. + claims, DDL or maintenance progress, and cleanup jobs carry `SessionRuntime` + and remain paired with their exact stable `SessionOperationEntry`. - accepted DDL and maintenance also retain a mandatory caller permit through terminal publication. - abandoned and terminal-rollback cleanup pair their active session entry with a mandatory internal permit; failed-precommit cleanup is covered by mandatory internal admission. -- weak upgrades used for admission rejection, handle drop, or exact terminal - resolution either register one of those authorities or stay within a bounded - section that cannot use components after rejection. +- foreground acquisition creates `AdmittedSessionRef` through + `SessionAdmission`, consumes it to create `AdmittedSessionRuntime`, validates + poison when required, and registers its stable operation or observer before + releasing admission and retaining plain `SessionRuntime`. +- terminal and cleanup paths reuse existing authority, upgrade the exact weak + state without new foreground admission, and validate both operation key and + transaction id directly on that state. - redo, mandatory-runtime, purge, file, and eviction workers are owned and - joined by their registered component owners rather than by `EngineRef`. + joined by their registered component owners. -An explicit shutdown may finish while a weak public handle's rejected upgrade -briefly retains an internal `Arc`. That handle has no admitted -authority to access components after rejection, so ordinary `Arc` reachability -is deliberately not a production shutdown condition. +A surviving public handle retains only a weak state reference and its small +closed admission façade. Once registry ownership is released it cannot retain +or recover component capabilities, so explicit shutdown and final owner drop +do not depend on destruction of public handles. The final reverse-order shutdown step releases `StorageRootLease`. A later engine can therefore acquire the root immediately after explicit shutdown, @@ -412,11 +432,13 @@ persistent `storage.lock` directory entry is never removed. After shutdown succeeds, `Engine` field order makes the final owner-drop sequence deterministic: -1. drop `Arc` +1. drop `Arc`, releasing the registry-owned session states and + their final `EngineCore` references 2. drop `ComponentRegistry` -Dropping `EngineInner` first releases the runtime-held quiescent guards before -registry-owned component owners start their final `QuiescentBox` drains. +Dropping `EngineInner` first releases `EngineCore` and its runtime-held +quiescent guards before registry-owned component owners start their final +`QuiescentBox` drains. `Engine::drop` invokes the same synchronous drain as `Engine::shutdown()`. An unintended owner drop can therefore block indefinitely while @@ -458,8 +480,11 @@ That provenance rule gives three guarantees: - stable owner identity survives cloning because guards keep the owner alive - page guards and arena state can rely on one exact pool provenance source -`PoolGuards` is only a named bundle of individually branded guards; it does not -weaken the single-owner provenance rule. +`EngineCore` owns one canonical `EnginePools` capability containing the four +typed pool handles and one prebuilt `PoolGuards` bundle. Session-coordinated +operations borrow that bundle through `SessionRuntime`; transaction attachments +do not clone it. `PoolGuards` remains only a named bundle of individually +branded guards and does not weaken the single-owner provenance rule. ## Arena And Page-Guard Lifetime Rules diff --git a/docs/lock-system.md b/docs/lock-system.md index c514834b..771ed181 100644 --- a/docs/lock-system.md +++ b/docs/lock-system.md @@ -467,8 +467,9 @@ completion. Every terminal user-transaction path drains its owner-local `OwnerLockState` before finishing the session transaction lifecycle. Transaction code mints one non-cloneable, transaction-id-bound `ReleasedTransactionLocks` proof only after -the local state is empty. Prepared and precommit paths also consume and drop -their retained lock-manager guard before minting the proof. +the local state is empty. Prepared and precommit paths reach the engine lock +manager through their retained terminal attachment, avoiding a second retained +component guard. `TrxAttachment::commit()` and `TrxAttachment::rollback()` consume a matching proof before they can make a running session idle or close an abandoned diff --git a/docs/public-error-audit.csv b/docs/public-error-audit.csv index 16c8184a..28323b6c 100644 --- a/docs/public-error-audit.csv +++ b/docs/public-error-audit.csv @@ -9,13 +9,13 @@ doradb-storage/src/error.rs,OperationOrRuntimeError::disclose,2 doradb-storage/src/error.rs,RuntimeOrFatalError::disclose,2 doradb-storage/src/error.rs,SharedFatalError::disclose,1 doradb-storage/src/log/mod.rs,LogSync::from_str,1 -doradb-storage/src/session.rs,Session::begin_trx,4 +doradb-storage/src/session.rs,Session::begin_trx,5 doradb-storage/src/session.rs,Session::buffer_pool_stats,1 doradb-storage/src/session.rs,Session::checkpoint_catalog,3 doradb-storage/src/session.rs,Session::checkpoint_catalog_and_truncate_redo_log,3 doradb-storage/src/session.rs,Session::checkpoint_table,5 doradb-storage/src/session.rs,Session::cleanup_secondary_mem_indexes,5 -doradb-storage/src/session.rs,Session::close,3 +doradb-storage/src/session.rs,Session::close,4 doradb-storage/src/session.rs,Session::create_index,9 doradb-storage/src/session.rs,Session::create_table,4 doradb-storage/src/session.rs,Session::drop_index,8 @@ -45,10 +45,10 @@ doradb-storage/src/table/access.rs,UserTableAccessor::update_known_cold_row,4 doradb-storage/src/table/access.rs,UserTableAccessor::update_known_hot_row,5 doradb-storage/src/table/access.rs,UserTableAccessor::update_unique_mvcc_input,18 doradb-storage/src/table/access.rs,UserTableAccessor::validate_table_mutation_update,1 -doradb-storage/src/trx/mod.rs,Transaction::commit,2 +doradb-storage/src/trx/mod.rs,Transaction::commit,1 doradb-storage/src/trx/mod.rs,Transaction::exec,2 doradb-storage/src/trx/mod.rs,Transaction::lock_table,2 -doradb-storage/src/trx/mod.rs,Transaction::rollback,3 +doradb-storage/src/trx/mod.rs,Transaction::rollback,2 doradb-storage/src/trx/stmt.rs,Statement::table_delete_unique_mvcc,3 doradb-storage/src/trx/stmt.rs,Statement::table_index_lookup_mvcc,2 doradb-storage/src/trx/stmt.rs,Statement::table_index_scan_mvcc,3 diff --git a/docs/tasks/000255-session-local-runtime-reachability.md b/docs/tasks/000255-session-local-runtime-reachability.md new file mode 100644 index 00000000..dc1b0476 --- /dev/null +++ b/docs/tasks/000255-session-local-runtime-reachability.md @@ -0,0 +1,300 @@ +--- +id: 000255 +title: Session-Local Runtime Reachability +status: implemented # proposal | implemented | superseded +created: 2026-08-04 +github_issue: 940 +--- + +# Task: Session-Local Runtime Reachability + +## Summary + +Replaced public session and transaction reachability through engine-wide weak +references with weak reachability to the exact `SessionState`. A successful +foreground admission now upgrades that session-local reference once, retains +the admission until stable operation ownership is registered, and resolves the +operation directly on the pinned state. + +Introduced one engine-owned `EngineCore` for shared runtime capabilities and a +strong `SessionRuntime` wrapper around the upgraded state. Operation pins, +observers, transaction attachments, mandatory work, and cleanup handoffs carry +that runtime instead of `EngineRef`. The final implementation removes +`EngineRef` and `WeakEngineRef` from storage source, removes normal +session-registry lookup from statement checkout, and borrows one canonical +pool-guard bundle. + +Lifecycle admission, shutdown authority, exact identity validation, public +APIs, persisted formats, and component teardown order remain unchanged. + +## Context + +`Issue Labels:` +`- type:task` +`- priority:medium` +`- codex` + +`Source Backlogs:` +`- docs/backlogs/000175-scalable-shared-resource-lifetime-management.md` + +`Related Tasks:` +`- docs/tasks/000247-statement-public-transaction-cancellation-ownership.md` +`- docs/tasks/000254-remove-engine-runtime-reference-accounting.md` + +`Benchmark Base:` +`- 2098cbb70316d383881aa3c05ba6ef56db408cc3` + +Task 000254 removed custom engine runtime-reference accounting, but ordinary +weak engine upgrades, lifecycle admission, session-registry lookup, and guard +cloning remained on session-coordinated paths. The registry already owned the +exact `SessionState`, and that state owned the stable operation slot used by +shutdown, so a per-session weak reference could remove the global lookup +without weakening the ownership proof. + +The original `TrxAttachment` also retained an engine reference, a strong +session state, and a cloned `PoolGuards` bundle. Those capabilities were +redundant once the strong session runtime could reach immutable engine +capabilities through the state. + +This was a bounded ownership-path refactor with no parent RFC. Backlog 000175 +remains open because it covers broader lifecycle admission, component guard, +buffer-page ownership, and shared-counter questions that this task did not +resolve. + +## Goals + +1. Make public `Session` and `Transaction` handles weakly reference their exact + `SessionState`. +2. Resolve session operations, observers, inspections, transaction checkout, + terminal paths, and cleanup directly on the upgraded state. +3. Introduce a reusable `EngineCore` and a strong `SessionRuntime` without a + strong cycle back to `EngineInner` or `SessionRegistry`. +4. Bind weak-state upgrade to successful lifecycle admission until stable + ownership is registered. +5. Make pins, attachments, accepted mandatory work, and cleanup jobs retain + `SessionRuntime`. +6. Centralize typed pool access and one canonical `PoolGuards` bundle. +7. Preserve exact operation-key and transaction-id validation, poison + behavior, terminal progress, cleanup, and shutdown wakeup rules. +8. Remove engine-wide weak upgrade, registry lookup, and attachment guard + cloning from transaction checkout. +9. Narrow single-capability interfaces, retaining `EngineCore` only for + multi-capability work, and measure no-op gains and any regressions. + +## Non-Goals + +1. The packed `EngineLifecycle` state and admission counter were not redesigned. +2. Operation-start admission and the shutdown-start race it closes remain. +3. General `Arc`, `Weak`, `QuiescentGuard`, and buffer-frame ownership were not + redesigned. +4. Component registration, worker lifetime, mandatory scheduling, recovery, + redo, undo, and persisted formats were not changed. +5. Public APIs, storage semantics, benchmark workloads, and CI timing policy + were not changed. +6. The contended `index-stream` regression was investigated but not fixed; + backlog 000175 remains open for that and wider lifetime work. + +## Plan + +The final ownership graph is: + +```text +Engine +└── Arc + ├── Arc ─ ─ weak ─ ─> SessionRegistry + ├── Arc ──> Arc ──> Arc + └── Arc + ▲ +SessionState ──> Arc ──> Arc + +Session / Transaction ─ ─ weak ─ ─> SessionState +operation authorities ──> SessionRuntime ──> Arc +``` + +`EngineInner` is the owner-facing coordination shell. `EngineCore` holds the +immutable component capabilities used by session-coordinated work, including +catalog, transaction system, table filesystem, lock manager, mandatory +runtime, poisoner, and `EnginePools`. Its registry back-reference is weak and +is used only for cold pointer-exact removal after a state becomes closed and +idle. + +`WeakSessionRef` combines `Weak` with the session admission +façade. Foreground acquisition follows one protocol: + +1. acquire `EngineAdmission` through `WeakSessionRef`; +2. consume the admitted reference to upgrade the exact state; +3. keep admission inside `AdmittedSessionRuntime`; +4. validate health and register the stable operation or observer proof; +5. consume the admitted runtime into `SessionRuntime`, releasing admission + before callbacks, blocking I/O, or `.await`. + +Terminal and cleanup paths use an explicit terminal upgrade without acquiring +new foreground admission, because shutdown must allow accepted ownership to +publish terminal state. Exact `SessionOperationKey` and `TrxID` checks remain +mandatory on both foreground and terminal paths. + +`SessionRuntime` is a typed wrapper around `Arc`. Direct state +methods own operation reservation, observer accounting, transaction checkout, +terminal publication, abandonment, and cleanup claims. `SessionRegistry` +remains the strong owner and shutdown traversal structure, but is absent from +normal resolution. + +`EnginePools` owns the four typed pool guards and one prebuilt `PoolGuards` +bundle. Session and transaction paths borrow the bundle. Interfaces that need +one capability receive the narrow type—for example staged runtime destruction +receives `&PoolGuards`, recovery resources are constructed from `PoolGuards`, +and owned current-index reads retain only the capabilities they use. +Multi-capability flows may receive `EngineCore`. + +`TrxAttachment`, session pins, mandatory guards, and cleanup work retain +`SessionRuntime`. Transaction foreground resolution is consolidated in +`Transaction::checkout`; terminal resolution is named +`checkout_terminal`. Prepared and precommit state derive required capabilities +from the attachment instead of cloning engine-core fields. + +State transitions publish under the session lifecycle mutex, release the +mutex, then perform pointer-exact registry removal and notification. This +preserves the listener-before-recheck shutdown protocol and prevents stale +removal from deleting a replacement state. + +## Implementation Notes + +Implemented session-local runtime reachability across engine, session, +transaction, DDL, catalog, table, recovery, index, log, and buffer-pool +consumers. Production storage source now contains neither `EngineRef` nor +`WeakEngineRef`; no replacement broad test handle was introduced. + +The admitted type-state was tightened during review. `WeakSessionRef` first +returns `AdmittedSessionRef`, whose consuming `upgrade` produces +`AdmittedSessionRuntime`. Admission therefore cannot be accidentally separated +from the weak upgrade or leaked as usable runtime authority, and it is consumed +only after stable ownership registration. + +Transaction review consolidated active resolution and core checkout into +`Transaction::checkout`, renamed terminal resolution to +`checkout_terminal`, and removed `let mut trx = self` rebinding patterns. +Runtime/core clones were audited: operation scopes borrow or move +`SessionRuntime`, prepared and precommit owners use their attachment, and +checkpoint, table-GC, and shutdown helpers borrow or consume only what their +lifetime requires. + +Capability review narrowed several interfaces: + +- CREATE TABLE helpers receive `EngineCore` only when they need multiple pools + or services; staged cleanup receives only `PoolGuards`. +- `RecoveryResources::new` accepts `PoolGuards` instead of `EnginePools`. +- `OwnedCurrentIndexReadHandle` retains only its required index-read + capabilities. +- Catalog, persistence, checkpoint, GC, logging, and test hooks capture narrow + guards. Test-only engine forwarding methods were removed; tests use + owner/core structure, narrow guards, or pre-created sessions. + +### Performance Verification + +Measurements used optimized builds, `--log-sync none`, fresh equivalent +storage roots, one warmup, and seven alternating samples per revision and +configuration on the same aarch64 Linux environment used for the preceding +lifetime tasks. + +| Workload | Baseline median ns | Candidate median ns | Latency delta | +| --- | ---: | ---: | ---: | +| `stmt-noop` 1 thread / 1 session | 73.524 | 47.865 | -34.899% | +| `stmt-noop` 4 threads / 16 sessions | 83.893 | 76.425 | -8.902% | +| `trx-noop` 1 thread / 1 session | 301.035 | 223.491 | -25.759% | +| `trx-noop` 4 threads / 16 sessions | 264.610 | 220.616 | -16.626% | +| unique `index-stream` 1/1 | 233,689 | 240,751 | +3.022% | +| unique `index-stream` 4/16 | 76,799 | 106,144 | +38.210% | +| non-unique `index-stream` 1/1 | 240,770 | 238,612 | -0.896% | +| non-unique `index-stream` 4/16 | 82,875 | 105,518 | +27.322% | + +Independent repeated 4/16 index-stream blocks reproduced the unfavorable +result. Paired `cargo flamegraph` profiles localized the added candidate CPU +to existing row-page and buffer-page reference-count operations: relaxed and +release `Arc` atomic helpers accounted for about 29.75% of candidate samples +versus 4.65% of baseline samples. Session-runtime and attachment pool-guard +access accounted for about 0.07%, and the removed engine weak upgrade, +registry lookup, and guard-bundle clone were absent. + +The original plan required no repeatable regression in the index-stream rows. +That acceptance rule was deliberately revised after profiling: the cause spans +index, row-page, and buffer-pool behavior and was not shown to be caused by the +session-runtime change. The user accepted deferring that investigation to +backlog 000175 rather than broadening this task without a root cause. + +### Verification + +- `rtk cargo check -p doradb-storage --tests`: passed. +- `rtk cargo build --workspace`: passed. +- `rtk cargo nextest run --workspace`: 1,646 passed. +- `rtk cargo clippy --workspace --all-targets -- -D warnings`: passed through + the style gate. +- `rtk cargo nextest run -p doradb-storage --no-default-features --features + libaio`: 1,553 passed. +- `rtk cargo clippy -p doradb-storage --no-default-features --features libaio + --all-targets -- -D warnings`: passed. +- `tools/style_audit.rs --diff-base origin/main`: passed for 37 branch-diff + Rust files. +- Focused line coverage across `engine.rs`, `session.rs`, and `trx/mod.rs` was + 95.84% (9,757/10,180): 97.21%, 95.96%, and 95.14% respectively. +- `rtk git diff --check`: passed. + +No parent RFC synchronization is required. Source backlog 000175 remains open +intentionally because only its session-coordinated reachability slice was +implemented. + +## Impacts + +- Session and transaction hot paths now use per-session weak reachability and + direct state resolution. +- `EngineCore` is the immutable shared capability boundary; `EngineInner` + retains owner orchestration. +- Session-coordinated runtime owners carry `SessionRuntime`, while + single-capability helpers use narrow guards. +- Pool guards are built once per engine and borrowed by session-coordinated + work. +- Shutdown, terminal cleanup, poison handling, and exact identity error + classifications remain compatible. +- Public APIs, dependencies, configuration, persisted formats, recovery + protocols, benchmark CLI, and CI policy are unchanged. +- The contended index-stream finding is a documented performance risk owned by + backlog 000175. + +## Test Cases + +1. Foreground admission either rejects shutdown or upgrades the exact state and + registers an operation or observer before admission release. +2. Transaction checkout and terminal paths validate the exact operation key + and transaction id directly on the state. +3. Stale operation, transaction, cleanup, and removal identities cannot affect + replacement state. +4. Healthy operations reject poison while poison-tolerant inspection remains + available during open admission. +5. Commit, rollback, abandonment, and cleanup publish terminal state after + foreground admission closes. +6. Public weak handles surviving shutdown neither retain components nor regain + usable runtime authority. +7. Explicit close, abandonment, observer release, and terminal publication + remove only pointer-identical idle state. +8. Accepted DDL, maintenance, precommit, cancellation, and cleanup retain the + runtime authority needed through their final handoff. +9. Typed pool access and canonical guard provenance remain correct across DDL, + recovery, table, and index paths. +10. Default and `libaio` suites preserve lifecycle, transaction, DDL, + persistence, checkpoint, recovery, poison, and storage-root behavior. +11. Structural review confirms no engine-wide weak upgrade, registry lookup, + attachment pool-guard clone, or separate core clone remains in transaction + checkout. +12. Paired no-op benchmarks improve, while the separately profiled index-stream + regression remains recoverably documented. + +## Open Questions + +No task-scoped design question remains. + +Backlog +[000175](../backlogs/000175-scalable-shared-resource-lifetime-management.md) +retains the unresolved engine-global admission cache line, broader +quiescent/component guard ownership, buffer-page reference-count contention, +and the measured choice between centralized, sharded, or retained counting. +Its future work should first reproduce the index-stream profile independently +and establish causality before changing index or buffer ownership. diff --git a/docs/tasks/next-id b/docs/tasks/next-id index aff09b82..bc96b9d5 100644 --- a/docs/tasks/next-id +++ b/docs/tasks/next-id @@ -1 +1 @@ -000255 +000256 diff --git a/docs/transaction-system.md b/docs/transaction-system.md index 89e158cc..cde89381 100644 --- a/docs/transaction-system.md +++ b/docs/transaction-system.md @@ -163,16 +163,19 @@ this runtime transaction contract. Each user statement runs through `Transaction::exec(async |stmt| { ... })`. The public `Transaction` is a weak, non-cloneable capability containing weak -engine reachability plus `SessionOperationKey` and its independent engine-wide -`TrxID`. `SessionOperationKey` is the exact `(SessionID, OperationID)` identity; +reachability to its exact `SessionState`, `SessionOperationKey`, and its +independent engine-wide `TrxID`. `SessionOperationKey` is the exact +`(SessionID, OperationID)` identity; the raw operation id is a session-local `u64` allocated from one sequence shared by transactions, DDL, maintenance, and explicit-lock mutation. The facade does -not own the crate-private `EngineRef`, `SessionState`, or stable operation -entry. Public transaction operations upgrade weak engine reachability, resolve -the exact operation key through the session registry, and build an -operation-local runtime attachment before callbacks or `.await` points. The -checkout or terminal claim then validates the handle's independent `TrxID` -against the entry under the entry mutex. +not own `EngineCore`, `SessionState`, a transaction core, or a stable operation +entry. A foreground checkout first acquires lifecycle admission through its +per-session façade, upgrades the exact weak state once, validates engine health, +and resolves the operation key directly on that state. It then builds a +`TrxAttachment` containing `SessionRuntime`; checkout validates the handle's +independent `TrxID` against the entry under the entry mutex. Terminal and +cleanup paths omit new foreground admission but perform the same exact state, +operation-key, and transaction-id validation. `SessionState` has orthogonal disposition (`Open`, `CloseRequested`, or `Abandoned`), one effectful operation slot (`Idle`, `Active`, or `Closed`), and @@ -227,9 +230,10 @@ During an active transaction, the owning box is checked out for one non-terminal operation through `SessionOperationCheckout`; ordinary checkout drop returns the same box through the entry mutex. This keeps repeated ownership transfers pointer-sized without allocating during statement execution. The -checkout owns the operation-local `TrxAttachment` and exposes a copyable -`TrxRuntime` value that pairs immutable `TrxContext` with runtime access to the -engine, pool guards, and session-local user-table cache. `TrxContext` never +checkout owns a `TrxAttachment` containing the exact `SessionRuntime` and +exposes a copyable `TrxRuntime` value that pairs immutable `TrxContext` with +borrowed access to `EngineCore`, its canonical pool guards, and the +session-local user-table cache. `TrxContext` never stores the attachment. Normal statement checkout/check-in does not reacquire the session lifecycle mutex, allocate, touch the operation change notifier, or send cleanup work. @@ -554,10 +558,12 @@ without assigning a commit timestamp. Terminal user-session completion is structurally gated by `ReleasedTransactionLocks`. Transaction code can mint this non-cloneable, transaction-id-bound proof only after draining the transaction's owner-local -lock state; prepared and precommit cleanup also drops its retained lock-manager -guard first. `TrxAttachment::commit()` and `TrxAttachment::rollback()` consume -and validate the proof before the session registry may make the session idle or -closed. +lock state through the engine lock manager reached from the retained terminal +attachment. `TrxAttachment::commit()` and `TrxAttachment::rollback()` consume +and validate the proof before direct state publication may make the session idle +or closed. If terminal publication leaves a close-requested or abandoned session +idle, `EngineCore` upgrades its weak registry back-reference and removes only +the pointer-identical registered state. For ordered commit, shared committed status is published before transaction locks are released, and session completion follows lock release. For rollback @@ -579,14 +585,17 @@ releasing lifecycle and explicit-lock ownership; shutdown then rescans for the first current blocker. A full traversal is required only to prove that no operation or observer remains. +Shutdown-discovered abandoned cleanup captures `SessionRuntime` from the +registered state before submission. The worker resolves the exact operation +directly on that state and never returns through the registry. + The nonblocking `try_shutdown()` uses the same first-blocker probe without installing an event. Consequently an ordinary open-session statement does not touch notification state, and an unobserved commit or rollback performs no notifier atomic update, event allocation, or wake. The listener-before-release protocol has no lost-wake interval, and `ShutdownBusy` remains observable while any operation, observer, mandatory caller permit, or mandatory internal permit -remains. `EngineRef` supplies shared component access but does not independently -block shutdown. +remains. Recovery only treats checkpoint metadata, table roots, and real redo headers as stable timestamp carriers. A no-log ordered commit has a volatile CTS that is diff --git a/doradb-storage/src/buffer/evict.rs b/doradb-storage/src/buffer/evict.rs index 6cf4ef70..fb4b3a30 100644 --- a/doradb-storage/src/buffer/evict.rs +++ b/doradb-storage/src/buffer/evict.rs @@ -2183,7 +2183,7 @@ pub(crate) mod tests { } fn owner_guard(&self) -> QuiescentGuard { - self.engine.inner().mem_pool.clone_inner() + self.engine.inner().pools.mem.clone() } fn shutdown(&self) { @@ -2195,7 +2195,7 @@ pub(crate) mod tests { type Target = EvictableBufferPool; fn deref(&self) -> &Self::Target { - &self.engine.inner().mem_pool + &self.engine.inner().pools.mem } } diff --git a/doradb-storage/src/buffer/readonly.rs b/doradb-storage/src/buffer/readonly.rs index eda34938..ccc9cdb6 100644 --- a/doradb-storage/src/buffer/readonly.rs +++ b/doradb-storage/src/buffer/readonly.rs @@ -3244,8 +3244,8 @@ pub(crate) mod tests { .unwrap(); let table_file = commit_table_file(&engine.inner().table_fs, table_file).await; - let capacity = engine.inner().disk_pool.capacity(); - let pool = engine.inner().disk_pool.clone_inner(); + let capacity = engine.inner().pools.disk.capacity(); + let pool = engine.inner().pools.disk.clone(); let base_page_id = 7u64; // Prepare one more block than cache capacity to force drop-only eviction. @@ -3266,10 +3266,7 @@ pub(crate) mod tests { let table_file = engine .inner() .table_fs - .open_table_file( - test_user_table_id(103), - engine.inner().disk_pool.clone_inner(), - ) + .open_table_file(test_user_table_id(103), engine.inner().pools.disk.clone()) .await .unwrap(); let pool_guard = pool.pool_guard(); @@ -3297,7 +3294,7 @@ pub(crate) mod tests { test_user_file_id(TableID::new(103)), BlockID::from(base_page_id + *i as u64), ); - engine.inner().disk_pool.try_get_frame_id(&key).is_some() + engine.inner().pools.disk.try_get_frame_id(&key).is_some() }) .count(); assert!(mapped_count < loaded_count); diff --git a/doradb-storage/src/catalog/checkpoint.rs b/doradb-storage/src/catalog/checkpoint.rs index 5269427d..9ff96580 100644 --- a/doradb-storage/src/catalog/checkpoint.rs +++ b/doradb-storage/src/catalog/checkpoint.rs @@ -388,7 +388,7 @@ impl MaintenanceExecutionSpec for CatalogCheckpointExecution { _resources: &mut Self::Resources, _panic_label: &mut Self::PanicLabel, ) -> CompletionResult { - let engine = scope.engine().clone(); + let engine = scope.engine(); let result = engine .catalog() .checkpoint_prepared(&engine.trx_sys) diff --git a/doradb-storage/src/catalog/history.rs b/doradb-storage/src/catalog/history.rs index 24758759..c1ba9cc9 100644 --- a/doradb-storage/src/catalog/history.rs +++ b/doradb-storage/src/catalog/history.rs @@ -670,7 +670,7 @@ mod tests { let temp_dir = TempDir::new().unwrap(); let engine = lightweight_test_engine(&temp_dir, "metadata_history").await; let table_id = create_table2_for_test(&engine).await; - let catalog = engine.catalog(); + let catalog = engine.inner().core.catalog(); let initial = catalog.resolve_user_table_current(table_id).unwrap(); let initial_cts = initial.effective_cts(); @@ -867,6 +867,8 @@ mod tests { TableRedoReplayFloor, ) { let current = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap(); @@ -958,7 +960,7 @@ mod tests { let temp_dir = TempDir::new().unwrap(); let engine = lightweight_test_engine(&temp_dir, "metadata_history_purge").await; let table_id = create_table2_for_test(&engine).await; - let catalog = engine.catalog(); + let catalog = engine.inner().core.catalog(); let mut ddl_session = engine.new_session().unwrap(); let index_no = ddl_session @@ -1018,6 +1020,8 @@ mod tests { .unwrap(); assert_eq!(usize::from(index_no), 1); let visible = engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, MAX_SNAPSHOT_TS) .unwrap(); @@ -1025,7 +1029,12 @@ mod tests { assert!(retained_metadata.idx.index_spec(1).is_some()); session.drop_index(table_id, index_no).await.unwrap(); - let table = engine.catalog().get_table_now(table_id).unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .unwrap(); assert!(table.layout_snapshot().secondary_indexes()[1].is_none()); assert!(!table.has_retired_secondary_indexes()); assert_eq!( @@ -1037,6 +1046,8 @@ mod tests { let recovered = lightweight_test_engine(&temp_dir, "metadata_history_recovery").await; let current = recovered + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap(); @@ -1061,6 +1072,8 @@ mod tests { assert!(!table.has_retired_secondary_indexes()); assert_eq!( recovered + .inner() + .core .catalog() .user_table_history_version_count(table_id), Some(0) diff --git a/doradb-storage/src/catalog/index.rs b/doradb-storage/src/catalog/index.rs index 7ffd6245..bedc47e3 100644 --- a/doradb-storage/src/catalog/index.rs +++ b/doradb-storage/src/catalog/index.rs @@ -1,9 +1,10 @@ use crate::buffer::{EvictableBufferPool, PoolGuard, PoolGuards}; +use crate::catalog::storage::CatalogStorage; use crate::catalog::{ Catalog, IndexColumnObject, IndexNo, IndexObject, IndexSpec, TableMetadata, TableObject, catalog_table_id_from_slot, }; -use crate::engine::EngineRef; +use crate::engine::EngineCore; use crate::error::{ CompletionErrorBridge, CompletionResult, DataIntegrityError, DataIntegrityResult, DiscloseResultExt, FatalError, OperationError, OperationOrRuntimeResult, OperationResult, @@ -19,6 +20,7 @@ use crate::index::{ }; use crate::log::redo::DDLRedo; use crate::obs; +use crate::poison::EnginePoisoner; use crate::quiescent::QuiescentGuard; use crate::row::RowRead; use crate::runtime::mandatory::{AcceptedExecution, MandatoryTaskMetadata, PreparedExecution}; @@ -454,15 +456,14 @@ struct CreateIndexRuntimeBuilder<'a> { impl<'a> CreateIndexRuntimeBuilder<'a> { #[inline] fn new( - engine: &EngineRef, - guards: &'a PoolGuards, + engine: &'a EngineCore, metadata: &'a TableMetadata, index_spec: &'a IndexSpec, build_ts: TrxID, ) -> Self { Self { - index_pool: engine.index_pool.clone_inner(), - index_guard: guards.index_guard(), + index_pool: engine.pools.index.clone(), + index_guard: engine.pool_guards().index_guard(), metadata, index_spec, build_ts, @@ -584,10 +585,6 @@ enum CreateIndexBuildPhase { } struct CreateIndexProgress { - // Accepted DDL is accounted by its mandatory caller permit and stable - // session operation; this handle supplies component access only. - engine: EngineRef, - guards: PoolGuards, table_id: TableID, index_no: IndexNo, build_ts: TrxID, @@ -599,17 +596,9 @@ struct CreateIndexProgress { impl CreateIndexProgress { #[inline] - fn new( - engine: EngineRef, - guards: PoolGuards, - table_id: TableID, - index_no: IndexNo, - trx: Transaction, - ) -> Self { + fn new(table_id: TableID, index_no: IndexNo, trx: Transaction) -> Self { let build_ts = trx.sts(); Self { - engine, - guards, table_id, index_no, build_ts, @@ -662,6 +651,7 @@ impl CreateIndexProgress { async fn execute_catalog_update( &mut self, + engine: &EngineCore, authority: PreparedCatalogWriteAuthority<'_>, metadata: &TableMetadata, index_spec: &IndexSpec, @@ -674,7 +664,7 @@ impl CreateIndexProgress { ) }); let res = execute_create_index_catalog_update( - &self.engine, + &engine.catalog().storage, trx, authority, self.table_id, @@ -686,13 +676,14 @@ impl CreateIndexProgress { match res { Ok(()) => Ok(()), Err(err) => { - self.rollback_before_catalog_commit().await?; + self.rollback_before_catalog_commit(engine.pool_guards()) + .await?; Err(RuntimeOrFatalError::from(err)) } } } - async fn commit_catalog(&mut self) -> RuntimeOrFatalResult { + async fn commit_catalog(&mut self, guards: &PoolGuards) -> RuntimeOrFatalResult { debug_assert_eq!(self.phase, CreateIndexBuildPhase::LayoutStaged); let trx = self.trx.take().unwrap_or_else(|| { panic!( @@ -706,7 +697,7 @@ impl CreateIndexProgress { Ok(cts) } Err(err) => { - self.cleanup_staged_runtime().await; + self.cleanup_staged_runtime(guards).await; self.phase = CreateIndexBuildPhase::Aborted; Err(err) } @@ -735,8 +726,11 @@ impl CreateIndexProgress { self.phase = CreateIndexBuildPhase::Installed; } - async fn rollback_before_catalog_commit(&mut self) -> RuntimeOrFatalResult<()> { - self.cleanup_staged_runtime().await; + async fn rollback_before_catalog_commit( + &mut self, + guards: &PoolGuards, + ) -> RuntimeOrFatalResult<()> { + self.cleanup_staged_runtime(guards).await; let rollback_res = rollback_active_ddl_trx(&mut self.trx).await; self.phase = CreateIndexBuildPhase::Aborted; rollback_res?; @@ -745,13 +739,14 @@ impl CreateIndexProgress { async fn cleanup_after_catalog_commit_failure( &mut self, + engine: &EngineCore, operation: &'static str, source: RuntimeOrFatalError, ) -> RuntimeOrFatalError { - self.cleanup_staged_runtime().await; + self.cleanup_staged_runtime(engine.pool_guards()).await; self.phase = CreateIndexBuildPhase::Aborted; poison_index_after_catalog_commit_with_source( - &self.engine, + &engine.poisoner, IndexDdlKind::Create, self.table_id, self.index_no, @@ -760,12 +755,12 @@ impl CreateIndexProgress { ) } - async fn cleanup_staged_runtime(&mut self) { + async fn cleanup_staged_runtime(&mut self, guards: &PoolGuards) { self.new_layout = None; if let Some(index) = self.staged_index.take() { // Preserve the existing best-effort cleanup policy. A destroy // failure is observed but does not replace the DDL source. - if let Err(report) = destroy_uninstalled_staged_index(index, &self.guards).await { + if let Err(report) = destroy_uninstalled_staged_index(index, guards).await { let report = report.attach(format!( "operation=cleanup_create_index_staged_runtime, table_id={}, index_no={}", self.table_id, self.index_no @@ -787,9 +782,6 @@ enum DropIndexBuildPhase { } struct DropIndexProgress { - // Accepted DDL is accounted by its mandatory caller permit and stable - // session operation; this handle supplies component access only. - engine: EngineRef, table_id: TableID, index_no: IndexNo, phase: DropIndexBuildPhase, @@ -799,9 +791,8 @@ struct DropIndexProgress { impl DropIndexProgress { #[inline] - fn new(engine: EngineRef, table_id: TableID, index_no: IndexNo, trx: Transaction) -> Self { + fn new(table_id: TableID, index_no: IndexNo, trx: Transaction) -> Self { Self { - engine, table_id, index_no, phase: DropIndexBuildPhase::LayoutStaged, @@ -819,6 +810,7 @@ impl DropIndexProgress { async fn execute_catalog_update( &mut self, + catalog: &Catalog, authority: PreparedCatalogWriteAuthority<'_>, old_index_spec: &IndexSpec, ) -> RuntimeOrFatalResult<()> { @@ -830,7 +822,7 @@ impl DropIndexProgress { ) }); let res = execute_drop_index_catalog_update( - &self.engine, + &catalog.storage, trx, authority, self.table_id, @@ -899,13 +891,14 @@ impl DropIndexProgress { async fn cleanup_after_catalog_commit_failure( &mut self, + poisoner: &EnginePoisoner, operation: &'static str, source: RuntimeOrFatalError, ) -> RuntimeOrFatalError { self.new_layout = None; self.phase = DropIndexBuildPhase::Aborted; poison_index_after_catalog_commit_with_source( - &self.engine, + poisoner, IndexDdlKind::Drop, self.table_id, self.index_no, @@ -1024,8 +1017,8 @@ impl AcceptedCreateIndex { let plan = self.plan.take().unwrap_or_else(|| { panic!("accepted CREATE INDEX invariant violated: execution plan is missing") }); - let engine = self.scope.engine().clone(); - let guards = self.scope.pool_guards(); + let engine = self.scope.engine().core(); + let guards = engine.pool_guards(); let table_id = plan.table_id; let index_no = plan.index_no; let index_no_usize = usize::from(index_no); @@ -1041,13 +1034,7 @@ impl AcceptedCreateIndex { err.attach("operation=create_index, phase=begin_private_transaction"), ) })?; - self.progress = Some(CreateIndexProgress::new( - engine.clone(), - guards.clone(), - table_id, - index_no, - trx, - )); + self.progress = Some(CreateIndexProgress::new(table_id, index_no, trx)); let progress = self .progress .as_mut() @@ -1067,7 +1054,7 @@ impl AcceptedCreateIndex { let key_validator = CreateIndexKeyValidator::new(&plan.new_index_spec); let collector = CreateIndexCollector::new( &plan.table, - &guards, + guards, plan.old_layout.as_ref(), &plan.new_index_spec, &plan.active_root, @@ -1085,7 +1072,7 @@ impl AcceptedCreateIndex { ) { Ok(runtime) => runtime, Err(err) => { - if let Err(cleanup) = progress.rollback_before_catalog_commit().await { + if let Err(cleanup) = progress.rollback_before_catalog_commit(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(cleanup)); } return Err(CompletionErrorBridge::capture(err)); @@ -1095,7 +1082,7 @@ impl AcceptedCreateIndex { let mut cold_rows = match collector.collect_current_cold().await { Ok(cold_rows) => cold_rows, Err(err) => { - if let Err(cleanup) = progress.rollback_before_catalog_commit().await { + if let Err(cleanup) = progress.rollback_before_catalog_commit(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(cleanup)); } return Err(CompletionErrorBridge::capture_operation_or_runtime(err)); @@ -1107,7 +1094,7 @@ impl AcceptedCreateIndex { .reach_phase(IndexDdlTestPhase::CreateColdCollectionComplete) .await; if let Err(err) = key_validator.prepare_cold(&mut cold_rows) { - if let Err(cleanup) = progress.rollback_before_catalog_commit().await { + if let Err(cleanup) = progress.rollback_before_catalog_commit(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(cleanup)); } return Err(CompletionErrorBridge::capture(err)); @@ -1116,7 +1103,7 @@ impl AcceptedCreateIndex { let cold_root = match build_create_index_disk_tree( &mut mutable_file, &disk_runtime, - &guards, + guards, &plan.new_index_spec, &cold_rows, build_ts, @@ -1125,7 +1112,7 @@ impl AcceptedCreateIndex { { Ok(root) => root, Err(err) => { - if let Err(cleanup) = progress.rollback_before_catalog_commit().await { + if let Err(cleanup) = progress.rollback_before_catalog_commit(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(cleanup)); } return Err(CompletionErrorBridge::capture_runtime_or_fatal(err)); @@ -1144,8 +1131,7 @@ impl AcceptedCreateIndex { ); let runtime_builder = CreateIndexRuntimeBuilder::new( - &engine, - &guards, + engine, plan.new_metadata.as_ref(), &plan.new_index_spec, build_ts, @@ -1153,7 +1139,7 @@ impl AcceptedCreateIndex { let mut hot_rows = match collector.collect_current_hot().await { Ok(hot_rows) => hot_rows, Err(err) => { - if let Err(cleanup) = progress.rollback_before_catalog_commit().await { + if let Err(cleanup) = progress.rollback_before_catalog_commit(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(cleanup)); } return Err(CompletionErrorBridge::capture(err)); @@ -1165,7 +1151,7 @@ impl AcceptedCreateIndex { .reach_phase(IndexDdlTestPhase::CreateHotCollectionComplete) .await; if let Err(err) = key_validator.prepare_hot(&mut hot_rows, &cold_rows) { - if let Err(cleanup) = progress.rollback_before_catalog_commit().await { + if let Err(cleanup) = progress.rollback_before_catalog_commit(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(cleanup)); } return Err(CompletionErrorBridge::capture(err)); @@ -1180,7 +1166,7 @@ impl AcceptedCreateIndex { match runtime_index { Ok(index) => progress.stage_runtime_index(index), Err(err) => { - if let Err(cleanup) = progress.rollback_before_catalog_commit().await { + if let Err(cleanup) = progress.rollback_before_catalog_commit(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(cleanup)); } return Err(CompletionErrorBridge::capture_operation_or_runtime(err)); @@ -1197,7 +1183,7 @@ impl AcceptedCreateIndex { .index_ddl_test .maybe_fail_create(CreateIndexTestFailure::AfterRuntimeStaged) { - if let Err(cleanup) = progress.rollback_before_catalog_commit().await { + if let Err(cleanup) = progress.rollback_before_catalog_commit(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(cleanup)); } return Err(CompletionErrorBridge::capture(err)); @@ -1213,7 +1199,12 @@ impl AcceptedCreateIndex { let authority = self.scope.catalog_write_authority(); if let Err(err) = progress - .execute_catalog_update(authority, plan.new_metadata.as_ref(), &plan.new_index_spec) + .execute_catalog_update( + engine, + authority, + plan.new_metadata.as_ref(), + &plan.new_index_spec, + ) .await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(err)); @@ -1224,7 +1215,7 @@ impl AcceptedCreateIndex { .reach_phase(IndexDdlTestPhase::CreateCatalogStaged) .await; let create_cts = progress - .commit_catalog() + .commit_catalog(guards) .await .map_err(CompletionErrorBridge::capture_runtime_or_fatal)?; #[cfg(test)] @@ -1241,6 +1232,7 @@ impl AcceptedCreateIndex { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress .cleanup_after_catalog_commit_failure( + engine, "table_root_publish", RuntimeOrFatalError::from(err), ) @@ -1267,11 +1259,11 @@ impl AcceptedCreateIndex { ) .is_none() { - progress.cleanup_staged_runtime().await; + progress.cleanup_staged_runtime(guards).await; progress.phase = CreateIndexBuildPhase::Aborted; return Err(CompletionErrorBridge::capture_runtime_or_fatal( poison_index_publication_invariant( - &engine, + &engine.poisoner, IndexDdlKind::Create, table_id, index_no, @@ -1398,8 +1390,8 @@ impl AcceptedDropIndex { let plan = self.plan.take().unwrap_or_else(|| { panic!("accepted DROP INDEX invariant violated: execution plan is missing") }); - let engine = self.scope.engine().clone(); - let guards = self.scope.pool_guards(); + let engine = self.scope.engine().core(); + let guards = engine.pool_guards(); let table_id = plan.table_id; let index_no = plan.index_no; let index_no_usize = usize::from(index_no); @@ -1415,12 +1407,7 @@ impl AcceptedDropIndex { err.attach("operation=drop_index, phase=begin_private_transaction"), ) })?; - self.progress = Some(DropIndexProgress::new( - engine.clone(), - table_id, - index_no, - trx, - )); + self.progress = Some(DropIndexProgress::new(table_id, index_no, trx)); let progress = self .progress .as_mut() @@ -1457,7 +1444,7 @@ impl AcceptedDropIndex { let authority = self.scope.catalog_write_authority(); if let Err(err) = progress - .execute_catalog_update(authority, &plan.old_index_spec) + .execute_catalog_update(engine.catalog(), authority, &plan.old_index_spec) .await { return Err(CompletionErrorBridge::capture_runtime_or_fatal(err)); @@ -1485,6 +1472,7 @@ impl AcceptedDropIndex { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress .cleanup_after_catalog_commit_failure( + &engine.poisoner, "table_root_publish", RuntimeOrFatalError::from(err), ) @@ -1513,7 +1501,12 @@ impl AcceptedDropIndex { { progress.phase = DropIndexBuildPhase::Aborted; return Err(CompletionErrorBridge::capture_runtime_or_fatal( - poison_index_publication_invariant(&engine, IndexDdlKind::Drop, table_id, index_no), + poison_index_publication_invariant( + &engine.poisoner, + IndexDdlKind::Drop, + table_id, + index_no, + ), )); } progress.mark_installed(); @@ -1525,10 +1518,10 @@ impl AcceptedDropIndex { engine.trx_sys.request_metadata_history_purge(); drop(plan.old_layout); - if let Err(err) = plan.table.cleanup_retired_secondary_indexes(&guards).await { + if let Err(err) = plan.table.cleanup_retired_secondary_indexes(guards).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal( poison_index_after_catalog_commit_with_source( - &engine, + &engine.poisoner, IndexDdlKind::Drop, table_id, index_no, @@ -1955,7 +1948,7 @@ async fn destroy_uninstalled_staged_index( #[inline] async fn execute_drop_index_catalog_update( - engine: &EngineRef, + storage: &CatalogStorage, trx: &mut Transaction, authority: PreparedCatalogWriteAuthority<'_>, table_id: TableID, @@ -1963,9 +1956,7 @@ async fn execute_drop_index_catalog_update( old_index_spec: &IndexSpec, ) -> RuntimeResult<()> { trx.stage_prepared_catalog_statement(authority, async |stmt| { - let deleted_columns = engine - .catalog() - .storage + let deleted_columns = storage .index_columns() .delete_by_index(stmt, table_id, index_no) .await?; @@ -1975,9 +1966,7 @@ async fn execute_drop_index_catalog_update( "drop-index catalog invariant violated: index-column delete count mismatch, table_id={table_id}, index_no={index_no}" ); - let index_deleted = engine - .catalog() - .storage + let index_deleted = storage .indexes() .delete_by_id(stmt, table_id, index_no) .await?; @@ -2005,7 +1994,7 @@ async fn execute_drop_index_catalog_update( /// inserted catalog primary key is therefore unique by construction. #[inline] async fn execute_create_index_catalog_update( - engine: &EngineRef, + storage: &CatalogStorage, trx: &mut Transaction, authority: PreparedCatalogWriteAuthority<'_>, table_id: TableID, @@ -2014,9 +2003,7 @@ async fn execute_create_index_catalog_update( index_spec: &IndexSpec, ) -> RuntimeResult<()> { trx.stage_prepared_catalog_statement(authority, async |stmt| { - let table_deleted = engine - .catalog() - .storage + let table_deleted = storage .tables() .delete_by_id(stmt, table_id) .await?; @@ -2025,9 +2012,7 @@ async fn execute_create_index_catalog_update( "create-index catalog invariant violated: validated table row is missing, table_id={table_id}" ); - engine - .catalog() - .storage + storage .tables() .insert( stmt, @@ -2038,9 +2023,7 @@ async fn execute_create_index_catalog_update( ) .await?; - engine - .catalog() - .storage + storage .indexes() .insert( stmt, @@ -2053,9 +2036,7 @@ async fn execute_create_index_catalog_update( .await?; for (index_column_no, index_key) in index_spec.cols.iter().enumerate() { - engine - .catalog() - .storage + storage .index_columns() .insert( stmt, @@ -2083,7 +2064,7 @@ async fn execute_create_index_catalog_update( #[inline] fn poison_index_after_catalog_commit_with_source( - engine: &EngineRef, + poisoner: &EnginePoisoner, kind: IndexDdlKind, table_id: TableID, index_no: IndexNo, @@ -2101,12 +2082,12 @@ fn poison_index_after_catalog_commit_with_source( "event=engine_poison component=catalog_index action=poison result=error error={:?}", report ); - RuntimeOrFatalError::from(engine.poisoner.poison(report).into_report()) + RuntimeOrFatalError::from(poisoner.poison(report).into_report()) } #[inline] fn poison_index_publication_invariant( - engine: &EngineRef, + poisoner: &EnginePoisoner, kind: IndexDdlKind, table_id: TableID, index_no: IndexNo, @@ -2122,7 +2103,7 @@ fn poison_index_publication_invariant( "event=engine_poison component=catalog_index action=poison result=error error={:?}", report ); - RuntimeOrFatalError::from(engine.poisoner.poison(report).into_report()) + RuntimeOrFatalError::from(poisoner.poison(report).into_report()) } #[cfg(test)] @@ -2340,6 +2321,8 @@ pub(crate) mod tests { metadata, .. } = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -2350,7 +2333,11 @@ pub(crate) mod tests { IndexDdlSnapshot { current_effective_cts: effective_cts, current_metadata: metadata, - history_count: engine.catalog().user_table_history_version_count(table_id), + history_count: engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), layout_generation: layout.generation(), runtime_slots: layout .secondary_indexes() @@ -2373,6 +2360,8 @@ pub(crate) mod tests { metadata, .. } = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -2382,7 +2371,11 @@ pub(crate) mod tests { assert_eq!(effective_cts, before.current_effective_cts); assert!(Arc::ptr_eq(&metadata, &before.current_metadata)); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), before.history_count ); let layout = table.layout_snapshot(); @@ -2683,6 +2676,8 @@ pub(crate) mod tests { assert_eq!(table.layout_snapshot().generation(), old_generation + 1); assert_eq!(active_secondary_root(&table, 1), SUPER_BLOCK_ID); let table_object = engine + .inner() + .core .catalog() .storage .tables() @@ -2754,6 +2749,8 @@ pub(crate) mod tests { assert!(layout.metadata().idx.index_spec(1).is_some()); assert!(layout.secondary_indexes()[1].is_some()); let CurrentTableState::Live { metadata, .. } = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -3039,7 +3036,7 @@ pub(crate) mod tests { )); publication_entered.recv_async().await.unwrap(); - let catalog = engine.new_ref().unwrap().catalog_guard(); + let catalog = engine.inner().core.catalog.clone(); let (started_tx, started_rx) = sync_channel(1); let (done_tx, done_rx) = sync_channel(1); let purge = spawn(move || { @@ -3061,6 +3058,8 @@ pub(crate) mod tests { let layout = table.layout_snapshot(); let CurrentTableState::Live { metadata, .. } = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -3589,7 +3588,13 @@ pub(crate) mod tests { .await .unwrap(); let table_id = table2(&engine).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); let row_id = insert_one_row( &table, @@ -3624,7 +3629,13 @@ pub(crate) mod tests { )) .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); assert_eq!(table.metadata().idx.next_index_no(), 2); assert!(table.metadata().idx.index_spec(1).is_some()); let session = engine.new_session().unwrap(); @@ -3653,7 +3664,13 @@ pub(crate) mod tests { .await .unwrap(); let table_id = table2(&engine).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); assert_eq!( @@ -3676,6 +3693,8 @@ pub(crate) mod tests { assert_eq!(root.secondary_index_roots.len(), 2); assert_eq!(root.secondary_index_roots[1], SUPER_BLOCK_ID); let catalog_indexes = engine + .inner() + .core .catalog() .storage .indexes() @@ -3697,7 +3716,13 @@ pub(crate) mod tests { let engine = Engine::bootstrap(lightweight_test_engine_config(main_dir, log_stem)) .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); assert_eq!(table.metadata().idx.next_index_no(), 2); assert!(table.metadata().idx.index_spec(1).is_none()); assert_eq!( @@ -3817,6 +3842,8 @@ pub(crate) mod tests { let mut old_session = engine.new_session().unwrap(); let old_trx = old_session.begin_trx().unwrap(); let retained_visible = engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, old_trx.sts()) .unwrap(); @@ -3866,6 +3893,8 @@ pub(crate) mod tests { let mut old_session = engine.new_session().unwrap(); let old_trx = old_session.begin_trx().unwrap(); let retained_visible = engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, old_trx.sts()) .unwrap(); @@ -3930,6 +3959,8 @@ pub(crate) mod tests { fn table_for_internal_assertion(engine: &Engine, table_id: TableID) -> Arc { engine + .inner() + .core .catalog() .get_table_now(table_id) .expect("test table should exist") @@ -4084,7 +4115,7 @@ pub(crate) mod tests { let mut session = engine.new_session().unwrap(); insert_one_row(&table, &mut session, vec![Val::from(1), Val::from("alpha")]).await; let before = index_ddl_snapshot(&engine, table_id, &table); - let allocated_before = engine.inner().index_pool.allocated(); + let allocated_before = engine.inner().pools.index.allocated(); engine .inner() @@ -4103,7 +4134,7 @@ pub(crate) mod tests { err.report().downcast_ref::().copied(), Some(RuntimeError::IndexAccess) ); - assert_eq!(engine.inner().index_pool.allocated(), allocated_before); + assert_eq!(engine.inner().pools.index.allocated(), allocated_before); assert_index_ddl_snapshot_unchanged(&before, &engine, table_id, &table); assert_eq!(table.metadata().idx.next_index_no(), 1); assert!(table.metadata().idx.index_spec(1).is_none()); diff --git a/doradb-storage/src/catalog/mod.rs b/doradb-storage/src/catalog/mod.rs index 79f38e76..279ac04a 100644 --- a/doradb-storage/src/catalog/mod.rs +++ b/doradb-storage/src/catalog/mod.rs @@ -1267,6 +1267,8 @@ pub(crate) mod tests { let (event_tx, event_rx) = flume::unbounded(); engine.inner().trx_sys.set_purge_test_observer(event_tx); if engine + .inner() + .core .catalog() .snapshot_dropped_table_file_cleanups() .iter() @@ -1282,6 +1284,8 @@ pub(crate) mod tests { PurgeTestEvent::DroppedTableStarted => dropped_table_started = true, PurgeTestEvent::CycleCompleted if dropped_table_started => { if engine + .inner() + .core .catalog() .snapshot_dropped_table_file_cleanups() .iter() @@ -1295,7 +1299,7 @@ pub(crate) mod tests { _ => {} } } - assert_dropped_table_floor(engine.catalog(), table_id); + assert_dropped_table_floor(engine.inner().core.catalog(), table_id); } /// Waits for targeted purge completion after dropped-table file cleanup becomes eligible. @@ -1306,6 +1310,8 @@ pub(crate) mod tests { let (event_tx, event_rx) = flume::unbounded(); engine.inner().trx_sys.set_purge_test_observer(event_tx); while engine + .inner() + .core .catalog() .retained_dropped_table_ids_now() .contains(&table_id) @@ -1320,7 +1326,7 @@ pub(crate) mod tests { } } } - assert_no_dropped_table_operational_state(engine.catalog(), table_id); + assert_no_dropped_table_operational_state(engine.inner().core.catalog(), table_id); } #[inline] @@ -1586,10 +1592,12 @@ pub(crate) mod tests { ) .await; engine + .inner() + .core .catalog() .next_table_id .store(USER_TABLE_ID_LIMIT.as_u64(), Ordering::SeqCst); - let _ = engine.catalog().next_table_id(); + let _ = engine.inner().core.catalog().next_table_id(); }); } @@ -1697,7 +1705,10 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir.clone(), Some("catalog-allocator")).await; - assert_eq!(engine.catalog().curr_next_table_id(), USER_TABLE_ID_START); + assert_eq!( + engine.inner().core.catalog().curr_next_table_id(), + USER_TABLE_ID_START + ); let mut session = engine.new_session().unwrap(); let table_spec = TableSpec::new(vec![ ColumnSpec::new("id", ValKind::I32, ColumnAttributes::empty()), @@ -1712,12 +1723,18 @@ pub(crate) mod tests { ), ]; let table_id1 = session.create_table(table_spec, index_specs).await.unwrap(); - assert_eq!(engine.catalog().curr_next_table_id(), table_id1 + 1); + assert_eq!( + engine.inner().core.catalog().curr_next_table_id(), + table_id1 + 1 + ); drop(session); drop(engine); let engine = open_catalog_test_engine(main_dir, Some("catalog-allocator")).await; - assert_eq!(engine.catalog().curr_next_table_id(), table_id1 + 1); + assert_eq!( + engine.inner().core.catalog().curr_next_table_id(), + table_id1 + 1 + ); let table_id2 = table1(&engine).await; assert!(table_id1 >= USER_TABLE_ID_START); assert_eq!(table_id2, table_id1 + 1); @@ -1749,7 +1766,13 @@ pub(crate) mod tests { ) .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); assert_eq!(table.metadata().idx.next_index_no(), 2); assert_eq!( table @@ -1773,7 +1796,13 @@ pub(crate) mod tests { drop(engine); let engine = open_catalog_test_engine(main_dir.clone(), Some(log_stem)).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); assert_eq!(table.metadata().idx.next_index_no(), 2); assert_eq!( table @@ -1784,12 +1813,14 @@ pub(crate) mod tests { 2 ); let indexes = engine + .inner() + .core .catalog() .storage .indexes() .list_uncommitted_by_table_id( &PoolGuards::builder() - .push(PoolRole::Meta, engine.inner().meta_pool.pool_guard()) + .push(PoolRole::Meta, engine.inner().pools.meta.pool_guard()) .build(), table_id, ) @@ -1812,7 +1843,13 @@ pub(crate) mod tests { drop(engine); let engine = open_catalog_test_engine(main_dir, Some(log_stem)).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); assert_eq!(table.metadata().idx.next_index_no(), 2); assert_eq!(table.metadata().idx.active_index_count(), 2); assert_eq!( @@ -1836,10 +1873,17 @@ pub(crate) mod tests { open_catalog_test_engine(temp_dir.path().to_path_buf(), Some("redo-floor-borrow")) .await; let table_id = table1(&engine).await; - let table = engine.catalog().get_table_now(table_id).unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .unwrap(); let owners_before = Arc::strong_count(&table); let (live, dropped) = engine + .inner() + .core .catalog() .snapshot_user_table_redo_floors(MIN_SNAPSHOT_TS); @@ -1861,7 +1905,7 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir, Some("catalog-checkpoint-now")).await; - let snap0 = engine.catalog().storage.checkpoint_snapshot(); + let snap0 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(snap0.catalog_replay_start_ts, MIN_SNAPSHOT_TS); assert!( snap0 @@ -1878,11 +1922,11 @@ pub(crate) mod tests { .checkpoint_catalog() .await .unwrap(); - let snap1 = engine.catalog().storage.checkpoint_snapshot(); + let snap1 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert!(snap1.catalog_replay_start_ts > MIN_SNAPSHOT_TS); assert_eq!( snap1.meta.next_table_id, - engine.catalog().curr_next_table_id() + engine.inner().core.catalog().curr_next_table_id() ); assert!( snap1 @@ -1905,7 +1949,7 @@ pub(crate) mod tests { .checkpoint_catalog() .await .unwrap(); - let snap2 = engine.catalog().storage.checkpoint_snapshot(); + let snap2 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(snap2.catalog_replay_start_ts, snap1.catalog_replay_start_ts); assert_eq!(snap2.meta.table_roots, snap1.meta.table_roots); }); @@ -1931,7 +1975,7 @@ pub(crate) mod tests { .await .unwrap(); - let snap = engine.catalog().storage.checkpoint_snapshot(); + let snap = engine.inner().core.catalog().storage.checkpoint_snapshot(); let root = snap .meta .table_roots @@ -1941,13 +1985,13 @@ pub(crate) mod tests { .expect("catalog checkpoint should publish at least one root"); let root_block_id = BlockID::from(root.root_block_id.unwrap().get()); let block_id = { - let disk_pool_guard = engine.catalog().storage.disk_pool.pool_guard(); + let disk_pool_guard = engine.inner().core.catalog().storage.disk_pool.pool_guard(); let index = ColumnBlockIndex::new( root_block_id, root.pivot_row_id, - engine.catalog().storage.mtb.file_kind(), - engine.catalog().storage.mtb.sparse_file(), - &engine.catalog().storage.disk_pool, + engine.inner().core.catalog().storage.mtb.file_kind(), + engine.inner().core.catalog().storage.mtb.sparse_file(), + &engine.inner().core.catalog().storage.disk_pool, &disk_pool_guard, ); let entry = index @@ -1993,7 +2037,7 @@ pub(crate) mod tests { .await .unwrap(); - let snap = engine.catalog().storage.checkpoint_snapshot(); + let snap = engine.inner().core.catalog().storage.checkpoint_snapshot(); let root = snap .meta .table_roots @@ -2003,13 +2047,13 @@ pub(crate) mod tests { .expect("catalog checkpoint should publish at least one root"); let root_block_id = BlockID::from(root.root_block_id.unwrap().get()); let entry = { - let disk_pool_guard = engine.catalog().storage.disk_pool.pool_guard(); + let disk_pool_guard = engine.inner().core.catalog().storage.disk_pool.pool_guard(); let index = ColumnBlockIndex::new( root_block_id, root.pivot_row_id, - engine.catalog().storage.mtb.file_kind(), - engine.catalog().storage.mtb.sparse_file(), - &engine.catalog().storage.disk_pool, + engine.inner().core.catalog().storage.mtb.file_kind(), + engine.inner().core.catalog().storage.mtb.sparse_file(), + &engine.inner().core.catalog().storage.disk_pool, &disk_pool_guard, ); index @@ -2054,7 +2098,7 @@ pub(crate) mod tests { .checkpoint_catalog() .await .unwrap(); - let snap1 = engine.catalog().storage.checkpoint_snapshot(); + let snap1 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert!(snap1.catalog_replay_start_ts > MIN_SNAPSHOT_TS); let roots_before = snap1.meta.table_roots; @@ -2074,7 +2118,7 @@ pub(crate) mod tests { .checkpoint_catalog() .await .unwrap(); - let snap2 = engine.catalog().storage.checkpoint_snapshot(); + let snap2 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert!(snap2.catalog_replay_start_ts > snap1.catalog_replay_start_ts); assert_eq!(snap2.meta.table_roots, roots_before); assert_eq!(snap2.meta.next_table_id, snap1.meta.next_table_id); @@ -2096,6 +2140,8 @@ pub(crate) mod tests { let trx_sys = &engine.inner().trx_sys; let batch1 = engine + .inner() + .core .catalog() .scan_checkpoint_batch( trx_sys.persisted_watermark_cts(), @@ -2110,14 +2156,18 @@ pub(crate) mod tests { ); let safe_cts_1 = batch1.safe_cts; engine + .inner() + .core .catalog() .apply_checkpoint_batch(batch1) .await .unwrap(); - let snap1 = engine.catalog().storage.checkpoint_snapshot(); + let snap1 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(snap1.catalog_replay_start_ts, safe_cts_1 + 1); let batch2 = engine + .inner() + .core .catalog() .scan_checkpoint_batch( trx_sys.persisted_watermark_cts(), @@ -2128,11 +2178,13 @@ pub(crate) mod tests { assert_eq!(batch2.catalog_ddl_txn_count, 0); assert_eq!(batch2.safe_cts, safe_cts_1); engine + .inner() + .core .catalog() .apply_checkpoint_batch(batch2) .await .unwrap(); - let snap2 = engine.catalog().storage.checkpoint_snapshot(); + let snap2 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(snap2.catalog_replay_start_ts, snap1.catalog_replay_start_ts); }); } @@ -2156,7 +2208,7 @@ pub(crate) mod tests { .checkpoint_catalog() .await .unwrap(); - let snap1 = engine.catalog().storage.checkpoint_snapshot(); + let snap1 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert!(snap1.catalog_replay_start_ts > MIN_SNAPSHOT_TS); let roots_before = snap1.meta.table_roots; @@ -2186,11 +2238,15 @@ pub(crate) mod tests { trx.commit().await.unwrap(); let checkpointed_table = engine + .inner() + .core .catalog() .get_table(checkpointed_table_id) .await .unwrap(); let replay_only_table = engine + .inner() + .core .catalog() .get_table(replay_only_table_id) .await @@ -2239,7 +2295,7 @@ pub(crate) mod tests { .checkpoint_catalog() .await .unwrap(); - let snap2 = engine.catalog().storage.checkpoint_snapshot(); + let snap2 = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert!(snap2.catalog_replay_start_ts > snap1.catalog_replay_start_ts); assert_eq!(snap2.meta.table_roots, roots_before); assert_eq!(snap2.meta.next_table_id, snap1.meta.next_table_id); diff --git a/doradb-storage/src/catalog/storage/columns.rs b/doradb-storage/src/catalog/storage/columns.rs index ae56b1c0..aed6169f 100644 --- a/doradb-storage/src/catalog/storage/columns.rs +++ b/doradb-storage/src/catalog/storage/columns.rs @@ -259,6 +259,8 @@ mod tests { let mut trx = session.begin_trx().unwrap(); trx.exec(async |stmt| { engine + .inner() + .core .catalog() .storage .columns() @@ -266,6 +268,8 @@ mod tests { .await .disclose()?; engine + .inner() + .core .catalog() .storage .columns() @@ -273,6 +277,8 @@ mod tests { .await .disclose()?; engine + .inner() + .core .catalog() .storage .columns() @@ -290,6 +296,8 @@ mod tests { trx.exec(async |stmt| { assert!( engine + .inner() + .core .catalog() .storage .columns() @@ -299,6 +307,8 @@ mod tests { ); assert!( !engine + .inner() + .core .catalog() .storage .columns() @@ -314,6 +324,8 @@ mod tests { trx.commit().await.unwrap(); let cols_42 = engine + .inner() + .core .catalog() .storage .columns() @@ -324,6 +336,8 @@ mod tests { assert_eq!(cols_42[0].column_no, 0); let cols_43 = engine + .inner() + .core .catalog() .storage .columns() @@ -337,6 +351,8 @@ mod tests { trx.exec(async |stmt| { assert!( !engine + .inner() + .core .catalog() .storage .columns() @@ -346,6 +362,8 @@ mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .columns() @@ -355,6 +373,8 @@ mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .columns() @@ -371,6 +391,8 @@ mod tests { assert!( engine + .inner() + .core .catalog() .storage .columns() @@ -381,6 +403,8 @@ mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .columns() @@ -431,6 +455,8 @@ mod tests { trx.exec(async |stmt| { for column in &columns { engine + .inner() + .core .catalog() .storage .columns() @@ -449,6 +475,8 @@ mod tests { trx.exec(async |stmt| { assert_eq!( engine + .inner() + .core .catalog() .storage .columns() @@ -459,6 +487,8 @@ mod tests { ); assert_eq!( engine + .inner() + .core .catalog() .storage .columns() @@ -476,6 +506,8 @@ mod tests { assert!( engine + .inner() + .core .catalog() .storage .columns() @@ -485,6 +517,8 @@ mod tests { .is_empty() ); let remaining = engine + .inner() + .core .catalog() .storage .columns() diff --git a/doradb-storage/src/catalog/storage/indexes.rs b/doradb-storage/src/catalog/storage/indexes.rs index 6f231775..c692f524 100644 --- a/doradb-storage/src/catalog/storage/indexes.rs +++ b/doradb-storage/src/catalog/storage/indexes.rs @@ -463,6 +463,8 @@ mod tests { let mut trx = session.begin_trx().unwrap(); trx.exec(async |stmt| { engine + .inner() + .core .catalog() .storage .indexes() @@ -470,6 +472,8 @@ mod tests { .await .disclose()?; engine + .inner() + .core .catalog() .storage .indexes() @@ -477,6 +481,8 @@ mod tests { .await .disclose()?; engine + .inner() + .core .catalog() .storage .indexes() @@ -494,6 +500,8 @@ mod tests { trx.exec(async |stmt| { assert!( engine + .inner() + .core .catalog() .storage .indexes() @@ -503,6 +511,8 @@ mod tests { ); assert!( !engine + .inner() + .core .catalog() .storage .indexes() @@ -518,6 +528,8 @@ mod tests { trx.commit().await.unwrap(); let idx_42 = engine + .inner() + .core .catalog() .storage .indexes() @@ -528,6 +540,8 @@ mod tests { assert_eq!(idx_42[0].index_no, 0); let idx_43 = engine + .inner() + .core .catalog() .storage .indexes() @@ -541,6 +555,8 @@ mod tests { trx.exec(async |stmt| { assert!( !engine + .inner() + .core .catalog() .storage .indexes() @@ -550,6 +566,8 @@ mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .indexes() @@ -559,6 +577,8 @@ mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .indexes() @@ -575,6 +595,8 @@ mod tests { assert!( engine + .inner() + .core .catalog() .storage .indexes() @@ -585,6 +607,8 @@ mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .indexes() @@ -629,6 +653,8 @@ mod tests { trx.exec(async |stmt| { for index in &indexes { engine + .inner() + .core .catalog() .storage .indexes() @@ -647,6 +673,8 @@ mod tests { trx.exec(async |stmt| { assert_eq!( engine + .inner() + .core .catalog() .storage .indexes() @@ -657,6 +685,8 @@ mod tests { ); assert_eq!( engine + .inner() + .core .catalog() .storage .indexes() @@ -674,6 +704,8 @@ mod tests { assert!( engine + .inner() + .core .catalog() .storage .indexes() @@ -683,6 +715,8 @@ mod tests { .is_empty() ); let remaining = engine + .inner() + .core .catalog() .storage .indexes() @@ -740,6 +774,8 @@ mod tests { trx.exec(async |stmt| { for index_column in &index_columns { engine + .inner() + .core .catalog() .storage .index_columns() @@ -758,6 +794,8 @@ mod tests { trx.exec(async |stmt| { assert_eq!( engine + .inner() + .core .catalog() .storage .index_columns() @@ -768,6 +806,8 @@ mod tests { ); assert_eq!( engine + .inner() + .core .catalog() .storage .index_columns() @@ -784,6 +824,8 @@ mod tests { trx.commit().await.unwrap(); let remaining_42 = engine + .inner() + .core .catalog() .storage .index_columns() @@ -797,6 +839,8 @@ mod tests { trx.exec(async |stmt| { assert_eq!( engine + .inner() + .core .catalog() .storage .index_columns() @@ -807,6 +851,8 @@ mod tests { ); assert_eq!( engine + .inner() + .core .catalog() .storage .index_columns() @@ -824,6 +870,8 @@ mod tests { assert!( engine + .inner() + .core .catalog() .storage .index_columns() @@ -833,6 +881,8 @@ mod tests { .is_empty() ); let remaining_43 = engine + .inner() + .core .catalog() .storage .index_columns() diff --git a/doradb-storage/src/catalog/storage/mod.rs b/doradb-storage/src/catalog/storage/mod.rs index e4d46190..02b11d3f 100644 --- a/doradb-storage/src/catalog/storage/mod.rs +++ b/doradb-storage/src/catalog/storage/mod.rs @@ -1575,7 +1575,7 @@ pub(crate) mod tests { let main_dir = temp_dir.path().to_path_buf(); let engine = open_catalog_test_engine(main_dir, Some(engine_name)).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let replay_start_ts = storage.checkpoint_snapshot().catalog_replay_start_ts; let batch = CatalogCheckpointBatch { replay_start_ts, @@ -1592,7 +1592,7 @@ pub(crate) mod tests { let err = expect_runtime_report( storage - .apply_checkpoint_batch(batch, engine.catalog().curr_next_table_id()) + .apply_checkpoint_batch(batch, engine.inner().core.catalog().curr_next_table_id()) .await .unwrap_err(), ); @@ -1616,7 +1616,7 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir, Some("catalog-empty-root-mismatch")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let mut snapshot = storage.checkpoint_snapshot(); let root = &mut snapshot.meta.table_roots[0]; assert_eq!(root.root_block_id, None); @@ -1661,7 +1661,7 @@ pub(crate) mod tests { let main_dir = temp_dir.path().to_path_buf(); let engine = open_catalog_test_engine(main_dir, Some("catalog-redo-table-range")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let replay_start_ts = storage.checkpoint_snapshot().catalog_replay_start_ts; let invalid_table_id = catalog_table_id_from_slot(CATALOG_TABLE_ROOT_DESC_COUNT); let batch = CatalogCheckpointBatch { @@ -1679,7 +1679,10 @@ pub(crate) mod tests { let err = expect_runtime_report( storage - .apply_checkpoint_batch(batch, engine.catalog().curr_next_table_id()) + .apply_checkpoint_batch( + batch, + engine.inner().core.catalog().curr_next_table_id(), + ) .await .unwrap_err(), ); @@ -1748,7 +1751,7 @@ pub(crate) mod tests { let main_dir = temp_dir.path().to_path_buf(); let engine = open_catalog_test_engine(main_dir, Some("catalog-lwc-direct-build")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let catalog_table = storage.get_catalog_table(TABLE_ID_COLUMNS).unwrap(); let metadata = catalog_table.metadata(); let table_id = USER_TABLE_ID_START + 101; @@ -1836,7 +1839,7 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir, Some("catalog-root-delete-deltas")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let table = storage.get_catalog_table(TABLE_ID_TABLES).unwrap(); let table_id = USER_TABLE_ID_START + 111; let rows = vec![RowRecord { @@ -1868,7 +1871,7 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir, Some("catalog-root-duplicate-pk")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let table = storage.get_catalog_table(TABLE_ID_TABLES).unwrap(); let table_id = USER_TABLE_ID_START + 112; let rows = vec![ @@ -1914,14 +1917,17 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir.clone(), Some("catalog-meta-reclaim")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let before_root = storage.mtb.active_root_unchecked(); let old_meta_block_id = before_root.meta_block_id; let before_allocated = before_root.alloc_map.allocated(); - apply_metadata_only_checkpoint(storage, engine.catalog().curr_next_table_id()) - .await - .unwrap(); + apply_metadata_only_checkpoint( + storage, + engine.inner().core.catalog().curr_next_table_id(), + ) + .await + .unwrap(); let after_root = storage.mtb.active_root_unchecked(); assert_ne!(after_root.meta_block_id, old_meta_block_id); @@ -1945,7 +1951,7 @@ pub(crate) mod tests { drop(engine); let engine = open_catalog_test_engine(main_dir, Some("catalog-meta-reclaim")).await; - let snap = engine.catalog().storage.checkpoint_snapshot(); + let snap = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(snap.catalog_replay_start_ts, expected_replay_start_ts); assert_eq!(snap.meta.next_table_id, USER_TABLE_ID_START); }); @@ -1967,7 +1973,7 @@ pub(crate) mod tests { .await .unwrap(); - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let before = storage.checkpoint_snapshot(); let marker = storage.publish_first_redo_log_seq(3).await.unwrap(); @@ -2005,7 +2011,7 @@ pub(crate) mod tests { .await .unwrap(); - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let snap = storage.checkpoint_snapshot(); let disk_pool_guard = storage.disk_pool.pool_guard(); let mut catalog_index_blocks = BTreeSet::new(); @@ -2026,21 +2032,25 @@ pub(crate) mod tests { for block_id in &catalog_index_blocks { let _ = engine .inner() - .disk_pool + .pools + .disk .invalidate_block(CATALOG_MTB_FILE_ID, *block_id); let key = BlockKey::new(CATALOG_MTB_FILE_ID, *block_id); - assert!(engine.inner().disk_pool.try_get_frame_id(&key).is_none()); + assert!(engine.inner().pools.disk.try_get_frame_id(&key).is_none()); } - let cached_before = engine.inner().disk_pool.allocated(); + let cached_before = engine.inner().pools.disk.allocated(); - apply_metadata_only_checkpoint(storage, engine.catalog().curr_next_table_id()) - .await - .unwrap(); + apply_metadata_only_checkpoint( + storage, + engine.inner().core.catalog().curr_next_table_id(), + ) + .await + .unwrap(); - assert_eq!(engine.inner().disk_pool.allocated(), cached_before); + assert_eq!(engine.inner().pools.disk.allocated(), cached_before); for block_id in catalog_index_blocks { let key = BlockKey::new(CATALOG_MTB_FILE_ID, block_id); - assert!(engine.inner().disk_pool.try_get_frame_id(&key).is_none()); + assert!(engine.inner().pools.disk.try_get_frame_id(&key).is_none()); } }); } @@ -2053,7 +2063,7 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir, Some("catalog-canceled-fast-path")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let before_root = storage.mtb.active_root_unchecked(); let old_meta_block_id = before_root.meta_block_id; let before_allocated = before_root.alloc_map.allocated(); @@ -2085,7 +2095,7 @@ pub(crate) mod tests { }; storage - .apply_checkpoint_batch(batch, engine.catalog().curr_next_table_id()) + .apply_checkpoint_batch(batch, engine.inner().core.catalog().curr_next_table_id()) .await .unwrap(); @@ -2111,7 +2121,7 @@ pub(crate) mod tests { open_catalog_test_engine(main_dir, Some("catalog-update-key-same-batch-insert")) .await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let table_id = USER_TABLE_ID_START + 5252; let batch = checkpoint_batch_with_ops( storage, @@ -2137,7 +2147,7 @@ pub(crate) mod tests { ); storage - .apply_checkpoint_batch(batch, engine.catalog().curr_next_table_id()) + .apply_checkpoint_batch(batch, engine.inner().core.catalog().curr_next_table_id()) .await .unwrap(); @@ -2159,7 +2169,7 @@ pub(crate) mod tests { let main_dir = temp_dir.path().to_path_buf(); let engine = open_catalog_test_engine(main_dir, Some("catalog-update-pk-column")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let table_id = USER_TABLE_ID_START + 6262; let batch = checkpoint_batch_with_ops( storage, @@ -2186,7 +2196,10 @@ pub(crate) mod tests { let err = expect_runtime_report( storage - .apply_checkpoint_batch(batch, engine.catalog().curr_next_table_id()) + .apply_checkpoint_batch( + batch, + engine.inner().core.catalog().curr_next_table_id(), + ) .await .unwrap_err(), ); @@ -2220,7 +2233,7 @@ pub(crate) mod tests { .await .unwrap(); - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let batch = checkpoint_batch_with_ops( storage, vec![CatalogRedoEntry { @@ -2236,7 +2249,7 @@ pub(crate) mod tests { ); storage - .apply_checkpoint_batch(batch, engine.catalog().curr_next_table_id()) + .apply_checkpoint_batch(batch, engine.inner().core.catalog().curr_next_table_id()) .await .unwrap(); @@ -2259,7 +2272,7 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir, Some("catalog-reclaim-invalid-root")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let active_before = storage.mtb.active_root_unchecked(); let active_meta_before = active_before.meta_block_id; let active_root_ts_before = active_before.root_ts; @@ -2277,7 +2290,7 @@ pub(crate) mod tests { MutableMultiTableFile::fork(&storage.mtb, storage.table_fs.background_writes()); mutable.apply_checkpoint_metadata( active_root_ts_before.saturating_add(1), - engine.catalog().curr_next_table_id(), + engine.inner().core.catalog().curr_next_table_id(), roots, ); let err = storage @@ -2305,7 +2318,7 @@ pub(crate) mod tests { open_catalog_test_engine(main_dir, Some("catalog-checkpoint-canceled-empty-root")) .await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let table = storage.get_catalog_table(TABLE_ID_TABLES).unwrap(); let root = CatalogTableRootDesc { table_id: TABLE_ID_TABLES, @@ -2357,7 +2370,7 @@ pub(crate) mod tests { .await .unwrap(); - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let table = storage.get_catalog_table(TABLE_ID_TABLES).unwrap(); let root = storage.checkpoint_snapshot().meta.table_roots[0]; assert!(root.root_block_id.is_some()); @@ -2404,14 +2417,16 @@ pub(crate) mod tests { .await .unwrap(); - let snap = engine.catalog().storage.checkpoint_snapshot(); + let snap = engine.inner().core.catalog().storage.checkpoint_snapshot(); let tables_root = snap.meta.table_roots[0]; let root_block_id = BlockID::from(tables_root.root_block_id.unwrap().get()); - let disk_pool_guard = engine.catalog().storage.disk_pool.pool_guard(); + let disk_pool_guard = engine.inner().core.catalog().storage.disk_pool.pool_guard(); - let cached_before_first = engine.inner().disk_pool.allocated(); + let cached_before_first = engine.inner().pools.disk.allocated(); let entries1 = engine + .inner() + .core .catalog() .storage .collect_index_entries(&disk_pool_guard, root_block_id) @@ -2419,25 +2434,28 @@ pub(crate) mod tests { .unwrap(); assert!(!entries1.is_empty()); - let cached_after_first = engine.inner().disk_pool.allocated(); + let cached_after_first = engine.inner().pools.disk.allocated(); assert!(cached_after_first >= cached_before_first); let root_key = BlockKey::new(CATALOG_MTB_FILE_ID, root_block_id); assert!( engine .inner() - .disk_pool + .pools + .disk .try_get_frame_id(&root_key) .is_some() ); let entries2 = engine + .inner() + .core .catalog() .storage .collect_index_entries(&disk_pool_guard, root_block_id) .await .unwrap(); assert_eq!(entries2.len(), entries1.len()); - assert_eq!(engine.inner().disk_pool.allocated(), cached_after_first); + assert_eq!(engine.inner().pools.disk.allocated(), cached_after_first); }); } @@ -2460,10 +2478,11 @@ pub(crate) mod tests { .await .unwrap(); - let snap1 = engine.catalog().storage.checkpoint_snapshot(); + let snap1 = engine.inner().core.catalog().storage.checkpoint_snapshot(); let tables_root1 = snap1.meta.table_roots[0]; assert!(tables_root1.root_block_id.is_some()); - assert_compact_catalog_root(&engine.catalog().storage, TABLE_ID_TABLES).await; + assert_compact_catalog_root(&engine.inner().core.catalog().storage, TABLE_ID_TABLES) + .await; let table2_id = table2(&engine).await; engine @@ -2473,14 +2492,23 @@ pub(crate) mod tests { .await .unwrap(); - let snap2 = engine.catalog().storage.checkpoint_snapshot(); + let snap2 = engine.inner().core.catalog().storage.checkpoint_snapshot(); let tables_root2 = snap2.meta.table_roots[0]; - let rows = - assert_compact_catalog_root(&engine.catalog().storage, TABLE_ID_TABLES).await; + let rows = assert_compact_catalog_root( + &engine.inner().core.catalog().storage, + TABLE_ID_TABLES, + ) + .await; assert!(tables_root2.root_block_id != tables_root1.root_block_id); assert_eq!(tables_root2.pivot_row_id, RowID::new(rows.len() as u64)); - let active_root = engine.catalog().storage.mtb.active_root_unchecked(); + let active_root = engine + .inner() + .core + .catalog() + .storage + .mtb + .active_root_unchecked(); let root_block_id1 = BlockID::from(tables_root1.root_block_id.unwrap().get()); let root_block_id2 = BlockID::from(tables_root2.root_block_id.unwrap().get()); assert!( @@ -2498,15 +2526,19 @@ pub(crate) mod tests { let recovered = open_catalog_test_engine(main_dir, Some("catalog-checkpoint-compact-rewrite")) .await; - let mut recovered_table_ids = recovered.catalog().list_user_table_ids_now(); + let mut recovered_table_ids = + recovered.inner().core.catalog().list_user_table_ids_now(); recovered_table_ids.sort(); let mut expected_table_ids = vec![table1_id, table2_id]; expected_table_ids.sort(); assert_eq!(recovered_table_ids, expected_table_ids); assert_eq!( - assert_compact_catalog_root(&recovered.catalog().storage, TABLE_ID_TABLES) - .await - .len(), + assert_compact_catalog_root( + &recovered.inner().core.catalog().storage, + TABLE_ID_TABLES + ) + .await + .len(), rows.len() ); }); @@ -2520,7 +2552,7 @@ pub(crate) mod tests { let engine = open_catalog_test_engine(main_dir, Some("catalog-compact-large-append")).await; - let storage = &engine.catalog().storage; + let storage = &engine.inner().core.catalog().storage; let table_id = USER_TABLE_ID_START + 9000; storage .apply_checkpoint_batch( @@ -2528,7 +2560,7 @@ pub(crate) mod tests { storage, vec![catalog_column_insert(table_id, 0, 30_000)], ), - engine.catalog().curr_next_table_id(), + engine.inner().core.catalog().curr_next_table_id(), ) .await .unwrap(); @@ -2552,7 +2584,7 @@ pub(crate) mod tests { storage .apply_checkpoint_batch( checkpoint_batch_with_ops(storage, second_batch), - engine.catalog().curr_next_table_id(), + engine.inner().core.catalog().curr_next_table_id(), ) .await .unwrap(); diff --git a/doradb-storage/src/catalog/storage/table_replay_silent_watermarks.rs b/doradb-storage/src/catalog/storage/table_replay_silent_watermarks.rs index 22ea3341..bb5d1b42 100644 --- a/doradb-storage/src/catalog/storage/table_replay_silent_watermarks.rs +++ b/doradb-storage/src/catalog/storage/table_replay_silent_watermarks.rs @@ -253,7 +253,7 @@ mod tests { let mut insert = engine.inner().trx_sys.begin_sys_trx(); insert .upsert_silent_watermark( - engine.catalog(), + engine.inner().core.catalog(), &guards, SilentWatermarkObject { table_id, @@ -282,7 +282,7 @@ mod tests { let mut update = engine.inner().trx_sys.begin_sys_trx(); update .upsert_silent_watermark( - engine.catalog(), + engine.inner().core.catalog(), &guards, SilentWatermarkObject { table_id, @@ -315,7 +315,7 @@ mod tests { let mut no_op = engine.inner().trx_sys.begin_sys_trx(); no_op .upsert_silent_watermark( - engine.catalog(), + engine.inner().core.catalog(), &guards, SilentWatermarkObject { table_id, diff --git a/doradb-storage/src/catalog/storage/tables.rs b/doradb-storage/src/catalog/storage/tables.rs index 5b58c79f..ad3e89cc 100644 --- a/doradb-storage/src/catalog/storage/tables.rs +++ b/doradb-storage/src/catalog/storage/tables.rs @@ -177,6 +177,8 @@ mod tests { let mut trx = session.begin_trx().unwrap(); trx.exec(async |stmt| { engine + .inner() + .core .catalog() .storage .tables() @@ -184,6 +186,8 @@ mod tests { .await .disclose()?; engine + .inner() + .core .catalog() .storage .tables() @@ -201,6 +205,8 @@ mod tests { trx.exec(async |stmt| { assert!( engine + .inner() + .core .catalog() .storage .tables() @@ -210,6 +216,8 @@ mod tests { ); assert!( !engine + .inner() + .core .catalog() .storage .tables() @@ -226,6 +234,8 @@ mod tests { assert!( engine + .inner() + .core .catalog() .storage .tables() @@ -236,6 +246,8 @@ mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .tables() @@ -260,10 +272,12 @@ mod tests { let table_id = table1(&engine).await; { let guards = PoolGuards::builder() - .push(PoolRole::Meta, engine.inner().meta_pool.pool_guard()) + .push(PoolRole::Meta, engine.inner().pools.meta.pool_guard()) .build(); assert!( engine + .inner() + .core .catalog() .storage .tables() diff --git a/doradb-storage/src/catalog/table.rs b/doradb-storage/src/catalog/table.rs index 83c7ff54..56160224 100644 --- a/doradb-storage/src/catalog/table.rs +++ b/doradb-storage/src/catalog/table.rs @@ -1,27 +1,32 @@ use crate::buffer::PoolGuards; use crate::catalog::spec::{ActiveIndexSpec, ColumnAttributes, ColumnSpec, IndexNo, IndexSpec}; +use crate::catalog::storage::CatalogStorage; use crate::catalog::{ - ColumnObject, IndexColumnObject, IndexObject, TableObject, catalog_table_id_from_slot, + Catalog, ColumnObject, IndexColumnObject, IndexObject, TableObject, catalog_table_id_from_slot, is_user_table, }; -use crate::engine::EngineRef; +use crate::component::EnginePools; +use crate::engine::EngineCore; use crate::error::{ CompletionErrorBridge, CompletionResult, FatalError, FatalResult, InternalError, InternalResult, IoResult, OperationError, OperationOrRuntimeResult, OperationResult, RuntimeError, RuntimeOrFatalError, RuntimeOrFatalResult, RuntimeResult, }; +use crate::file::fs::FileSystem; use crate::file::table_file::{MutableTableFile, TableFile}; use crate::id::{TableID, TrxID}; use crate::index::BlockIndex; use crate::log::redo::DDLRedo; use crate::map::FastHashSet; use crate::obs; +use crate::poison::EnginePoisoner; use crate::row::ops::SelectKey; use crate::row::{Row, RowRead}; use crate::runtime::mandatory::{AcceptedExecution, MandatoryTaskMetadata, PreparedExecution}; use crate::serde::{Deser, DeserResult, MinBytesHint, Ser, Serde, min_bytes_hint}; use crate::session::{AcceptedDdlScope, PreparedDdlScope}; use crate::table::{Table, TableRedoReplayFloor}; +use crate::trx::sys::TransactionSystem; use crate::trx::{PreparedCatalogWriteAuthority, Transaction}; use crate::value::{Val, ValKind, ValType}; use error_stack::{Report, ResultExt}; @@ -248,7 +253,7 @@ impl CreateTableProgress { } #[inline] - async fn publish_file(&mut self, engine: &EngineRef) -> RuntimeResult<()> { + async fn publish_file(&mut self, trx_sys: &TransactionSystem) -> RuntimeResult<()> { debug_assert_eq!(self.phase, CreateTablePhase::CatalogStaged); let root_ts = self .trx @@ -262,8 +267,7 @@ impl CreateTableProgress { let CreateTableFile::Mutable(mutable_file) = file else { panic!("create-table file is mutable before publish"); }; - let table_file = engine - .trx_sys + let table_file = trx_sys .publish_table_file_root(mutable_file, root_ts, true) .await .change_context(RuntimeError::CatalogAccess) @@ -279,11 +283,7 @@ impl CreateTableProgress { } #[inline] - async fn build_runtime( - &mut self, - guards: &PoolGuards, - engine: &EngineRef, - ) -> RuntimeResult<()> { + async fn build_runtime(&mut self, pools: &EnginePools) -> RuntimeResult<()> { debug_assert_eq!(self.phase, CreateTablePhase::FilePublished); let Some(CreateTableFile::Published(table_file)) = self.file.as_ref() else { panic!("published table file is present before runtime build"); @@ -291,8 +291,8 @@ impl CreateTableProgress { let table_file = Arc::clone(table_file); let active_root = table_file.active_root_unchecked(); let blk_idx = BlockIndex::new( - engine.meta_pool.clone_inner(), - guards.meta_guard(), + pools.meta.clone(), + pools.pool_guards().meta_guard(), active_root.pivot_row_id, active_root.column_block_index_root, ) @@ -306,13 +306,13 @@ impl CreateTableProgress { })?; let table = Arc::new( Table::new( - engine.mem_pool.clone_inner(), - engine.index_pool.clone_inner(), - guards.index_guard(), + pools.mem.clone(), + pools.index.clone(), + pools.pool_guards().index_guard(), self.table_id, blk_idx, table_file, - engine.disk_pool.clone_inner(), + pools.disk.clone(), ) .await .change_context(RuntimeError::CatalogAccess) @@ -341,7 +341,7 @@ impl CreateTableProgress { } #[inline] - fn install_runtime(&mut self, engine: &EngineRef, create_cts: TrxID) -> bool { + fn install_runtime(&mut self, catalog: &Catalog, create_cts: TrxID) -> bool { debug_assert_eq!(self.phase, CreateTablePhase::CatalogCommitted); let table = Arc::clone( self.staged_table @@ -350,7 +350,7 @@ impl CreateTableProgress { ); // The table id was atomically allocated and this DDL owns the metadata // gate through commit, so no cache entry can exist for this runtime. - if !engine.catalog().insert_user_table(create_cts, table) { + if !catalog.insert_user_table(create_cts, table) { self.phase = CreateTablePhase::Aborted; return false; } @@ -360,7 +360,7 @@ impl CreateTableProgress { } #[inline] - fn delete_provisional_file(&mut self, engine: &EngineRef) -> IoResult<()> { + fn delete_provisional_file(&mut self, table_fs: &FileSystem) -> IoResult<()> { match self.file.take() { Some(CreateTableFile::Mutable(mutable_file)) => { let _ = mutable_file.try_delete(); @@ -368,7 +368,7 @@ impl CreateTableProgress { Some(CreateTableFile::Published(table_file)) => drop(table_file), None => {} } - engine.table_fs.delete_user_table_file(self.table_id) + table_fs.delete_user_table_file(self.table_id) } async fn destroy_staged_runtime(&mut self, guards: &PoolGuards) -> RuntimeResult<()> { @@ -388,16 +388,15 @@ impl CreateTableProgress { async fn abort_before_catalog_commit( &mut self, - engine: &EngineRef, - guards: &PoolGuards, + engine: &EngineCore, operation: &'static str, source: Report, ) -> RuntimeOrFatalError { let source_debug = format!("{source:?}"); let mut cleanup_error = None; - if let Err(err) = self.destroy_staged_runtime(guards).await { + if let Err(err) = self.destroy_staged_runtime(engine.pool_guards()).await { cleanup_error = Some(poison_error_source( - engine, + &engine.poisoner, RuntimeOrFatalError::from(err), FatalError::Poisoned, format!( @@ -412,7 +411,7 @@ impl CreateTableProgress { && cleanup_error.is_none() { cleanup_error = Some(poison_error_source( - engine, + &engine.poisoner, err, FatalError::RollbackAccess, format!( @@ -421,7 +420,7 @@ impl CreateTableProgress { ), )); } - if let Err(err) = self.delete_provisional_file(engine) + if let Err(err) = self.delete_provisional_file(&engine.table_fs) && cleanup_error.is_none() { cleanup_error = Some(RuntimeOrFatalError::from( @@ -438,16 +437,15 @@ impl CreateTableProgress { async fn abort_after_root_publish_commit_error( &mut self, - engine: &EngineRef, - guards: &PoolGuards, + engine: &EngineCore, operation: &'static str, source: RuntimeOrFatalError, ) -> RuntimeOrFatalError { let source_debug = format!("{source:?}"); - if let Err(err) = self.destroy_staged_runtime(guards).await { + if let Err(err) = self.destroy_staged_runtime(engine.pool_guards()).await { self.phase = CreateTablePhase::Aborted; return poison_error_source( - engine, + &engine.poisoner, RuntimeOrFatalError::from(err), FatalError::Poisoned, format!( @@ -458,7 +456,7 @@ impl CreateTableProgress { } self.phase = CreateTablePhase::Aborted; poison_error_source( - engine, + &engine.poisoner, source, FatalError::Poisoned, format!( @@ -1350,8 +1348,7 @@ impl AcceptedCreateTable { .progress .as_mut() .unwrap_or_else(|| panic!("accepted CREATE progress exists during execution")); - let engine = scope.engine().clone(); - let guards = scope.pool_guards(); + let engine = scope.engine(); let table_id = progress.table_id; #[cfg(test)] @@ -1377,7 +1374,7 @@ impl AcceptedCreateTable { Err(err) => { let source_debug = format!("{err:?}"); progress.phase = CreateTablePhase::Aborted; - if let Err(cleanup_err) = progress.delete_provisional_file(&engine) { + if let Err(cleanup_err) = progress.delete_provisional_file(&engine.table_fs) { return Err(CompletionErrorBridge::capture(cleanup_err.attach(format!( "create table provisional-file cleanup failed after transaction begin: table_id={table_id}, source_error={source_debug}" )))); @@ -1398,7 +1395,7 @@ impl AcceptedCreateTable { let catalog_objects = progress.take_catalog_objects(); let authority = scope.catalog_write_authority(); let exec_res = execute_create_table_catalog_staging( - &engine, + &engine.catalog().storage, progress .trx .as_mut() @@ -1411,7 +1408,7 @@ impl AcceptedCreateTable { if let Err(err) = exec_res { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress - .abort_before_catalog_commit(&engine, &guards, "catalog_staging", err) + .abort_before_catalog_commit(engine, "catalog_staging", err) .await, )); } @@ -1430,20 +1427,15 @@ impl AcceptedCreateTable { { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress - .abort_before_catalog_commit( - &engine, - &guards, - "test_after_catalog_staging", - err, - ) + .abort_before_catalog_commit(engine, "test_after_catalog_staging", err) .await, )); } - if let Err(err) = progress.publish_file(&engine).await { + if let Err(err) = progress.publish_file(&engine.trx_sys).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress - .abort_before_catalog_commit(&engine, &guards, "file_publish", err) + .abort_before_catalog_commit(engine, "file_publish", err) .await, )); } @@ -1461,15 +1453,15 @@ impl AcceptedCreateTable { { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress - .abort_before_catalog_commit(&engine, &guards, "test_after_file_publish", err) + .abort_before_catalog_commit(engine, "test_after_file_publish", err) .await, )); } - if let Err(err) = progress.build_runtime(&guards, &engine).await { + if let Err(err) = progress.build_runtime(&engine.pools).await { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress - .abort_before_catalog_commit(&engine, &guards, "runtime_build", err) + .abort_before_catalog_commit(engine, "runtime_build", err) .await, )); } @@ -1487,7 +1479,7 @@ impl AcceptedCreateTable { { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress - .abort_before_catalog_commit(&engine, &guards, "test_after_runtime_build", err) + .abort_before_catalog_commit(engine, "test_after_runtime_build", err) .await, )); } @@ -1495,19 +1487,14 @@ impl AcceptedCreateTable { #[cfg(test)] engine .table_ddl_test - .maybe_poison_before_create_commit(&engine); + .maybe_poison_before_create_commit(&engine.poisoner); let create_cts = match progress.commit_catalog().await { Ok(create_cts) => create_cts, Err(err) => { return Err(CompletionErrorBridge::capture_runtime_or_fatal( progress - .abort_after_root_publish_commit_error( - &engine, - &guards, - "catalog_commit", - err, - ) + .abort_after_root_publish_commit_error(engine, "catalog_commit", err) .await, )); } @@ -1520,7 +1507,7 @@ impl AcceptedCreateTable { .await; assert!( - progress.install_runtime(&engine, create_cts), + progress.install_runtime(engine.catalog(), create_cts), "allocated CREATE TABLE id duplicated current runtime during accepted execution: table_id={table_id}" ); @@ -1657,7 +1644,7 @@ impl AcceptedDropTable { .progress .as_mut() .unwrap_or_else(|| panic!("accepted DROP progress exists during execution")); - let engine = scope.engine().clone(); + let engine = scope.engine(); let table_id = progress.plan.table_id; let table = progress.plan.take_table(); @@ -1720,7 +1707,7 @@ impl AcceptedDropTable { let metadata = table.metadata().clone(); let authority = scope.catalog_write_authority(); let exec_res = execute_drop_table_catalog_cascade( - &engine, + &engine.catalog().storage, progress .trx .as_mut() @@ -1745,7 +1732,7 @@ impl AcceptedDropTable { } return Err(CompletionErrorBridge::capture_runtime_or_fatal( poison_error_source( - &engine, + &engine.poisoner, RuntimeOrFatalError::from(err), FatalError::Poisoned, format!( @@ -1771,7 +1758,7 @@ impl AcceptedDropTable { Err(err) => { return Err(CompletionErrorBridge::capture_runtime_or_fatal( poison_error_source( - &engine, + &engine.poisoner, err, FatalError::Poisoned, format!( @@ -1792,7 +1779,7 @@ impl AcceptedDropTable { let replay_floor = engine .catalog() .effective_user_table_redo_replay_floor(table_id, table.redo_replay_floor_snapshot()); - finish_drop_table_runtime_retention(&engine, table_id, table, drop_cts, replay_floor) + finish_drop_table_runtime_retention(engine, table_id, table, drop_cts, replay_floor) .map_err(CompletionErrorBridge::capture)?; progress.phase = DropTablePhase::RuntimeRetained; @@ -1837,8 +1824,7 @@ pub(crate) fn reject_non_user_table_id( /// Ensure the user-table catalog row exists for a DDL operation. #[inline] pub(crate) async fn ensure_user_table_catalog_row( - guards: &PoolGuards, - engine: &EngineRef, + engine: &EngineCore, table_id: TableID, operation: &'static str, ) -> OperationOrRuntimeResult<()> { @@ -1846,7 +1832,7 @@ pub(crate) async fn ensure_user_table_catalog_row( .catalog() .storage .tables() - .find_uncommitted_by_id(guards, table_id) + .find_uncommitted_by_id(engine.pool_guards(), table_id) .await? .is_some() { @@ -1859,8 +1845,7 @@ pub(crate) async fn ensure_user_table_catalog_row( /// Return the validated runtime table for an index-DDL target. pub(crate) async fn validated_index_ddl_target( - guards: &PoolGuards, - engine: &EngineRef, + engine: &EngineCore, table_id: TableID, operation: &'static str, ) -> OperationOrRuntimeResult> { @@ -1870,7 +1855,7 @@ pub(crate) async fn validated_index_ddl_target( .validate_user_table_live(table_id) .await .attach_with(|| format!("operation={operation}"))?; - ensure_user_table_catalog_row(guards, engine, table_id, operation).await?; + ensure_user_table_catalog_row(engine, table_id, operation).await?; Ok(table) } @@ -1906,7 +1891,7 @@ fn reject_user_table_primary_key_indexes( /// namespace, so catalog insert Operation failures are invariant violations. #[inline] async fn execute_create_table_catalog_staging( - engine: &EngineRef, + storage: &CatalogStorage, trx: &mut Transaction, authority: PreparedCatalogWriteAuthority<'_>, table_id: TableID, @@ -1919,33 +1904,16 @@ async fn execute_create_table_catalog_staging( index_columns, } = catalog_objects; trx.stage_prepared_catalog_statement(authority, async |stmt| { - engine - .catalog() - .storage - .tables() - .insert(stmt, &table) - .await?; + storage.tables().insert(stmt, &table).await?; for column_object in columns { - engine - .catalog() - .storage - .columns() - .insert(stmt, &column_object) - .await?; + storage.columns().insert(stmt, &column_object).await?; } for index_object in indexes { - engine - .catalog() - .storage - .indexes() - .insert(stmt, &index_object) - .await?; + storage.indexes().insert(stmt, &index_object).await?; } for index_column_object in index_columns { - engine - .catalog() - .storage + storage .index_columns() .insert(stmt, &index_column_object) .await?; @@ -1967,34 +1935,26 @@ async fn execute_create_table_catalog_staging( #[inline] async fn execute_drop_table_catalog_cascade( - engine: &EngineRef, + storage: &CatalogStorage, trx: &mut Transaction, authority: PreparedCatalogWriteAuthority<'_>, table_id: TableID, metadata: &TableMetadata, ) -> RuntimeResult<()> { trx.stage_prepared_catalog_statement(authority, async |stmt| { - let index_columns_deleted = engine - .catalog() - .storage + let index_columns_deleted = storage .index_columns() .delete_by_table_id(stmt, table_id) .await?; - let indexes_deleted = engine - .catalog() - .storage + let indexes_deleted = storage .indexes() .delete_by_table_id(stmt, table_id) .await?; - let columns_deleted = engine - .catalog() - .storage + let columns_deleted = storage .columns() .delete_by_table_id(stmt, table_id) .await?; - let table_deleted = engine - .catalog() - .storage + let table_deleted = storage .tables() .delete_by_id(stmt, table_id) .await?; @@ -2002,9 +1962,7 @@ async fn execute_drop_table_catalog_cascade( table_deleted, "drop-table catalog invariant violated: validated table row is missing, table_id={table_id}" ); - engine - .catalog() - .storage + storage .table_replay_silent_watermarks() .delete_by_table_id(stmt, table_id) .await?; @@ -2059,7 +2017,7 @@ fn assert_drop_catalog_delete_counts( #[inline] fn finish_drop_table_runtime_retention( - engine: &EngineRef, + engine: &EngineCore, table_id: TableID, table: Arc
, drop_cts: TrxID, @@ -2073,7 +2031,7 @@ fn finish_drop_table_runtime_retention( return Ok(()); } Err(poison_drop_table_after_gate( - engine, + &engine.poisoner, table_id, "runtime_retention", )) @@ -2081,7 +2039,7 @@ fn finish_drop_table_runtime_retention( #[inline] fn poison_drop_table_after_gate( - engine: &EngineRef, + poisoner: &EnginePoisoner, table_id: TableID, operation: &'static str, ) -> Report { @@ -2096,13 +2054,13 @@ fn poison_drop_table_after_gate( "event=engine_poison component=catalog_table action=poison result=error error={:?}", report ); - engine.poisoner.poison(report).into_report() + poisoner.poison(report).into_report() } /// Fatalizes a typed catalog source while retaining its physical evidence. #[inline] fn poison_error_source( - engine: &EngineRef, + poisoner: &EnginePoisoner, source: RuntimeOrFatalError, reason: FatalError, message: String, @@ -2112,7 +2070,7 @@ fn poison_error_source( "event=engine_poison component=catalog_table action=poison result=error error={:?}", report ); - RuntimeOrFatalError::from(engine.poisoner.poison(report).into_report()) + RuntimeOrFatalError::from(poisoner.poison(report).into_report()) } #[inline] @@ -2256,12 +2214,11 @@ pub(crate) mod tests { } #[inline] - pub(super) fn maybe_poison_before_create_commit(&self, engine: &EngineRef) { + pub(super) fn maybe_poison_before_create_commit(&self, poisoner: &EnginePoisoner) { if *self.create_failure.lock() == Some(CreateTableTestFailure::PoisonBeforeCatalogCommit) { - let _ = engine - .poisoner + let _ = poisoner .poison(Report::new(FatalError::Poisoned).attach("forced create-table poison")); } } @@ -2358,24 +2315,39 @@ pub(crate) mod tests { } fn assert_no_user_table_publication(engine: &Engine, table_id: TableID) { - assert!(engine.catalog().get_table_now(table_id).is_none()); assert!( engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .is_none() + ); + assert!( + engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .is_none() ); assert!( engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, MAX_SNAPSHOT_TS) .is_none() ); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), None ); - assert_no_dropped_table_operational_state(engine.catalog(), table_id); + assert_no_dropped_table_operational_state(engine.inner().core.catalog(), table_id); } struct TableDdlSnapshot { @@ -2395,6 +2367,8 @@ pub(crate) mod tests { metadata, table: current_table, } = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -2405,8 +2379,14 @@ pub(crate) mod tests { effective_cts, metadata, table: current_table, - history_count: engine.catalog().user_table_history_version_count(table_id), + history_count: engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), retained_dropped_state: engine + .inner() + .core .catalog() .retained_dropped_table_ids_now() .contains(&table_id), @@ -2427,6 +2407,8 @@ pub(crate) mod tests { metadata, table: current_table, } = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -2437,11 +2419,17 @@ pub(crate) mod tests { assert!(Arc::ptr_eq(&metadata, &before.metadata)); assert!(Arc::ptr_eq(¤t_table, &before.table)); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), before.history_count ); assert_eq!( engine + .inner() + .core .catalog() .retained_dropped_table_ids_now() .contains(&table_id), @@ -3026,9 +3014,19 @@ pub(crate) mod tests { let verify_session = engine.new_session().unwrap(); let guards = verify_session.pool_guards(); for table_id in [table_id1, table_id2] { - assert!(engine.catalog().get_table(table_id).await.is_some()); assert!( engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_some() + ); + assert!( + engine + .inner() + .core .catalog() .storage .tables() @@ -3039,6 +3037,8 @@ pub(crate) mod tests { ); assert!( !engine + .inner() + .core .catalog() .storage .columns() @@ -3049,6 +3049,8 @@ pub(crate) mod tests { ); assert!( !engine + .inner() + .core .catalog() .storage .indexes() @@ -3059,6 +3061,8 @@ pub(crate) mod tests { ); assert!( !engine + .inner() + .core .catalog() .storage .index_columns() @@ -3087,7 +3091,7 @@ pub(crate) mod tests { .unwrap(); let mut session = engine.new_session().unwrap(); let session_id = session.id(); - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let table_file_path = engine.inner().table_fs.user_table_file_path(table_id); let err = session @@ -3135,7 +3139,7 @@ pub(crate) mod tests { .await .unwrap(); let mut session = engine.new_session().unwrap(); - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let table_file_path = engine.inner().table_fs.user_table_file_path(table_id); let err = session @@ -3171,7 +3175,7 @@ pub(crate) mod tests { .unwrap(); let mut session = engine.new_session().unwrap(); let session_id = session.id(); - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let table_file_path = engine.inner().table_fs.user_table_file_path(table_id); let (table_spec, index_specs) = drop_table_test_spec(); @@ -3213,7 +3217,7 @@ pub(crate) mod tests { .await .unwrap(); let mut session = engine.new_session().unwrap(); - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let table_file_path = engine.inner().table_fs.user_table_file_path(table_id); let hook = Arc::new(FailingFirstWriteHook::new(table_file_path.clone())); let _install = install_storage_backend_test_hook(hook.clone()); @@ -3256,7 +3260,7 @@ pub(crate) mod tests { .await .unwrap(); let mut session = engine.new_session().unwrap(); - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let table_file_path = engine.inner().table_fs.user_table_file_path(table_id); let (table_spec, index_specs) = drop_table_test_spec(); @@ -3288,7 +3292,7 @@ pub(crate) mod tests { .await .unwrap(); let mut session = engine.new_session().unwrap(); - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let table_file_path = engine.inner().table_fs.user_table_file_path(table_id); let (table_spec, index_specs) = drop_table_test_spec(); @@ -3320,7 +3324,7 @@ pub(crate) mod tests { .await .unwrap(); let mut session = engine.new_session().unwrap(); - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let table_file_path = engine.inner().table_fs.user_table_file_path(table_id); let (table_spec, index_specs) = drop_table_test_spec(); @@ -3390,10 +3394,9 @@ pub(crate) mod tests { LockDebugEntryState::Granted, )); - let engine_ref = engine.new_ref().unwrap(); + let mut writer_session = engine.new_session().unwrap(); let (owner_tx, owner_rx) = flume::bounded(1); let writer = smol::spawn(async move { - let mut writer_session = engine_ref.new_session().unwrap(); let mut writer_trx = writer_session.begin_trx().unwrap(); owner_tx .send_async(trx_tests::lock_owner(&writer_trx).unwrap()) @@ -3568,7 +3571,7 @@ pub(crate) mod tests { let blocker = LockOwner::transaction(SessionID::new(91_301), TrxID::new(91_301)); assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), LockResource::TableData(table_id), LockMode::Exclusive, blocker, @@ -3609,6 +3612,8 @@ pub(crate) mod tests { .await; assert_eq!( engine + .inner() + .core .lock_manager() .release(LockResource::TableData(table_id), blocker), 1 @@ -3672,7 +3677,7 @@ pub(crate) mod tests { let blocker = LockOwner::transaction(SessionID::new(91_302), TrxID::new(91_302)); assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), LockResource::TableData(table_id), LockMode::Exclusive, blocker, @@ -3717,6 +3722,8 @@ pub(crate) mod tests { ); assert_eq!( engine + .inner() + .core .lock_manager() .release(LockResource::TableData(table_id), blocker), 1 @@ -3738,10 +3745,9 @@ pub(crate) mod tests { .await .unwrap(); - let engine_ref = engine.new_ref().unwrap(); + let mut writer_session = engine.new_session().unwrap(); let (owner_tx, owner_rx) = flume::bounded(1); let external_writer = smol::spawn(async move { - let mut writer_session = engine_ref.new_session().unwrap(); let mut writer_trx = writer_session.begin_trx().unwrap(); owner_tx .send_async(trx_tests::lock_owner(&writer_trx).unwrap()) @@ -3904,6 +3910,8 @@ pub(crate) mod tests { corrupt_trx .exec(async |stmt| { let deleted = engine + .inner() + .core .catalog() .storage .tables() @@ -3940,7 +3948,15 @@ pub(crate) mod tests { ); assert_eq!(table.lifecycle.inspect_terminal(), TableTerminal::Dropping); assert_checkpoint_workflow_closed(&table); - assert!(engine.catalog().get_table(table_id).await.is_some()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_some() + ); assert_eq!(active_operation_count(&engine.inner().session_registry), 1); let shutdown_err = engine.try_shutdown().unwrap_err(); assert_eq!( @@ -3963,7 +3979,7 @@ pub(crate) mod tests { smol::block_on(async { let temp_dir = TempDir::new().unwrap(); let engine = lightweight_test_engine(&temp_dir, "redo_testsys_lightweight").await; - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let mut session = engine.new_session().unwrap(); let session_id = session.id(); let (table_spec, index_specs) = drop_table_test_spec(); @@ -4034,7 +4050,15 @@ pub(crate) mod tests { assert!(rendered.contains(&format!("table_id={table_id}"))); assert_table_ddl_snapshot_unchanged(&before, &engine, table_id, &table); - assert!(engine.catalog().get_table(table_id).await.is_some()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_some() + ); assert!(has_lock_entry( &engine, owner, @@ -4091,7 +4115,15 @@ pub(crate) mod tests { let mut other_session = engine.new_session().unwrap(); other_session.drop_table(other_table_id).await.unwrap(); - assert!(engine.catalog().get_table(other_table_id).await.is_none()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(other_table_id) + .await + .is_none() + ); assert_eq!(table.lifecycle.inspect_terminal(), TableTerminal::Dropping); drop(root_lease); @@ -4129,7 +4161,15 @@ pub(crate) mod tests { .await .unwrap(); assert_ne!(created_table_id, table_id); - assert!(engine.catalog().get_table(created_table_id).await.is_some()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(created_table_id) + .await + .is_some() + ); assert!( Path::new( &engine @@ -4223,7 +4263,7 @@ pub(crate) mod tests { smol::block_on(async { let temp_dir = TempDir::new().unwrap(); let engine = lightweight_test_engine(&temp_dir, "redo_testsys_lightweight").await; - let table_id = engine.catalog().curr_next_table_id(); + let table_id = engine.inner().core.catalog().curr_next_table_id(); let (entered, release) = engine .inner() .table_ddl_test @@ -4245,7 +4285,14 @@ pub(crate) mod tests { .lock_table(table_id, TableLockMode::Shared) .await .unwrap(); - assert!(engine.catalog().get_table_now(table_id).is_some()); + assert!( + engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .is_some() + ); verify_session.unlock_table(table_id).unwrap(); verify_session.drop_table(table_id).await.unwrap(); assert!(engine.inner().poisoner.poison_error().is_none()); @@ -4292,7 +4339,7 @@ pub(crate) mod tests { let temp_dir = TempDir::new().unwrap(); let engine = lightweight_test_engine(&temp_dir, "redo_testsys_lightweight").await; - let create_table_id = engine.catalog().curr_next_table_id(); + let create_table_id = engine.inner().core.catalog().curr_next_table_id(); let (create_entered, create_release) = engine .inner() .table_ddl_test @@ -4563,7 +4610,7 @@ pub(crate) mod tests { let horizon = horizon_session.begin_trx().unwrap(); let mut drop_session = engine.new_session().unwrap(); drop_session.drop_table(table_id).await.unwrap(); - assert_dropped_table_runtime(engine.catalog(), table_id); + assert_dropped_table_runtime(engine.inner().core.catalog(), table_id); let mut lock_session = engine.new_session().unwrap(); let session_owner = LockOwner::session_explicit(lock_session.id()); @@ -4674,9 +4721,19 @@ pub(crate) mod tests { session_id, LockResource::TableData(table_id), )); - assert!(engine.catalog().get_table(table_id).await.is_none()); assert!( engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_none() + ); + assert!( + engine + .inner() + .core .catalog() .storage .tables() @@ -4687,6 +4744,8 @@ pub(crate) mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .columns() @@ -4697,6 +4756,8 @@ pub(crate) mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .indexes() @@ -4707,6 +4768,8 @@ pub(crate) mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .index_columns() @@ -4717,6 +4780,8 @@ pub(crate) mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .tables() @@ -4727,6 +4792,8 @@ pub(crate) mod tests { ); assert!( !engine + .inner() + .core .catalog() .storage .columns() @@ -4802,23 +4869,40 @@ pub(crate) mod tests { session.drop_table(table_id).await.unwrap(); let drop_cts = session.last_cts(); assert_eq!( - engine.catalog().retained_dropped_table_ids_now(), + engine + .inner() + .core + .catalog() + .retained_dropped_table_ids_now(), vec![table_id] ); session.wait_for_gc_horizon_after(drop_cts).await.unwrap(); request_and_wait_for_purge_cycle(&engine, &event_rx).await; assert!( engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .is_none() ); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), None ); - assert_dropped_table_floor(engine.catalog(), table_id); - assert!(engine.catalog().get_table_now(table_id).is_none()); + assert_dropped_table_floor(engine.inner().core.catalog(), table_id); + assert!( + engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .is_none() + ); assert!(Path::new(&table_file_path).exists()); engine @@ -4829,10 +4913,19 @@ pub(crate) mod tests { .unwrap(); wait_for_no_dropped_table_operational_state(&engine, table_id).await; assert!(!Path::new(&table_file_path).exists()); - assert!(engine.catalog().retained_dropped_table_ids_now().is_empty()); - assert_no_dropped_table_operational_state(engine.catalog(), table_id); assert!( engine + .inner() + .core + .catalog() + .retained_dropped_table_ids_now() + .is_empty() + ); + assert_no_dropped_table_operational_state(engine.inner().core.catalog(), table_id); + assert!( + engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, MAX_SNAPSHOT_TS) .is_none() @@ -5028,6 +5121,8 @@ pub(crate) mod tests { session.drop_table(table_id).await.unwrap(); let trx_sys = &engine.inner().trx_sys; let batch = engine + .inner() + .core .catalog() .scan_checkpoint_batch( trx_sys.persisted_watermark_cts(), @@ -5081,6 +5176,8 @@ pub(crate) mod tests { .unwrap(); assert!( engine + .inner() + .core .catalog() .storage .checkpoint_snapshot() @@ -5098,7 +5195,15 @@ pub(crate) mod tests { )) .await .unwrap(); - assert!(engine.catalog().get_table(table_id).await.is_none()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_none() + ); assert!(!Path::new(&table_file_path).exists()); }); } diff --git a/doradb-storage/src/component.rs b/doradb-storage/src/component.rs index 9207c53e..2654c227 100644 --- a/doradb-storage/src/component.rs +++ b/doradb-storage/src/component.rs @@ -494,7 +494,7 @@ pool_access_newtype!(IndexPool, EvictableBufferPool); pool_access_newtype!(MemPool, EvictableBufferPool); pool_access_newtype!(DiskPool, ReadonlyBufferPool); -/// Inner buffer-pool handles used by engine startup and recovery. +/// Canonical engine buffer-pool capability shared by session runtime work. pub(crate) struct EnginePools { /// Metadata pool used for catalog and block-index pages. pub(crate) meta: QuiescentGuard, @@ -504,6 +504,20 @@ pub(crate) struct EnginePools { pub(crate) mem: QuiescentGuard, /// Readonly persisted-page pool. pub(crate) disk: QuiescentGuard, + /// Prebuilt guards for the exact four pool identities above. + guards: PoolGuards, +} + +impl Clone for EnginePools { + #[inline] + fn clone(&self) -> Self { + Self::new( + self.meta.clone(), + self.index.clone(), + self.mem.clone(), + self.disk.clone(), + ) + } } impl EnginePools { @@ -515,23 +529,25 @@ impl EnginePools { mem: QuiescentGuard, disk: QuiescentGuard, ) -> Self { + let guards = PoolGuards::builder() + .push(PoolRole::Meta, meta.pool_guard()) + .push(PoolRole::Index, index.pool_guard()) + .push(PoolRole::Mem, mem.pool_guard()) + .push(PoolRole::Disk, disk.pool_guard()) + .build(); Self { meta, index, mem, disk, + guards, } } - /// Build a full guard bundle for all engine buffer pools. + /// Borrow the canonical guard bundle for all engine buffer pools. #[inline] - pub(crate) fn pool_guards(&self) -> PoolGuards { - PoolGuards::builder() - .push(PoolRole::Meta, self.meta.pool_guard()) - .push(PoolRole::Index, self.index.pool_guard()) - .push(PoolRole::Mem, self.mem.pool_guard()) - .push(PoolRole::Disk, self.disk.pool_guard()) - .build() + pub(crate) fn pool_guards(&self) -> &PoolGuards { + &self.guards } } diff --git a/doradb-storage/src/engine.rs b/doradb-storage/src/engine.rs index dbe04956..aa1ee3f1 100644 --- a/doradb-storage/src/engine.rs +++ b/doradb-storage/src/engine.rs @@ -4,8 +4,8 @@ //! including start, stop, recover, and execute commands. See //! `docs/engine-component-lifetime.md` for the runtime-versus-owner lifetime //! model that this module enforces with the component registry. -use crate::buffer::PoolRole; use crate::buffer::SharedPoolEvictorWorkers; +use crate::buffer::{PoolGuards, PoolRole}; #[cfg(test)] use crate::catalog::index::tests::IndexDdlTestController; #[cfg(test)] @@ -20,7 +20,7 @@ use crate::error::{ ConfigError, DiscloseError, DiscloseResultExt, LifecycleError, LifecycleResult, Result, }; use crate::file::fs::{FileSystem, FileSystemWorkers}; -use crate::id::{SessionID, SessionOperationKey, TrxID}; +use crate::id::SessionID; use crate::lock::LockManager; use crate::obs; use crate::poison::EnginePoisoner; @@ -28,7 +28,7 @@ use crate::quiescent::QuiescentGuard; use crate::root::{StorageRootLease, StorageRootLeaseAttempt}; use crate::runtime::block_on; use crate::runtime::mandatory::{MandatoryRuntime, MandatoryRuntimeWorkers}; -use crate::session::{Session, SessionRegistry}; +use crate::session::{Session, SessionAdmission, SessionCleanupRequest, SessionRegistry}; #[cfg(test)] use crate::table::tests::MaintenanceTestController; use crate::trx::SessionOperationState; @@ -101,7 +101,8 @@ impl ShutdownOrigin { } } -struct EngineLifecycle { +/// Packed engine-wide operation admission and shutdown coordination. +pub(crate) struct EngineLifecycle { /// Packed lifecycle state and active admission count. /// /// Bits `[0, LIFECYCLE_STATE_BITS)` store [`EngineLifecycleState`]. The @@ -137,8 +138,9 @@ impl EngineLifecycle { .unwrap_or_else(|state| panic!("invalid engine lifecycle state: {state}")) } + /// Acquire one operation-start admission while the engine is running. #[inline] - fn admit(&self) -> LifecycleResult> { + pub(crate) fn admit(&self) -> LifecycleResult> { loop { let word = self.state.load(Ordering::Acquire); let state = EngineLifecycleState::try_from(word & LIFECYCLE_STATE_MASK) @@ -230,9 +232,15 @@ impl EngineLifecycle { /// Registers for the transition away from the running state. #[inline] - fn shutdown_listener(&self) -> EventListener { + pub(crate) fn shutdown_listener(&self) -> EventListener { self.shutdown_started.listen() } + + /// Returns whether owner-side shutdown has started. + #[inline] + pub(crate) fn shutdown_started(&self) -> bool { + self.inspect_state() != EngineLifecycleState::Running + } } /// Short-lived proof that an operation entered while the engine was running. @@ -308,35 +316,13 @@ impl Engine { let inner = self.inner(); inner.with_admitted_operation(|| { let id = inner.next_session_id(); + let admission = Arc::new(SessionAdmission::new(Arc::clone(&inner.lifecycle))); inner .session_registry - .create_session(inner, EngineRef::new(Arc::clone(inner)), id) + .create_session(Arc::clone(&inner.core), admission, id) }) } - /// Return the shared catalog handle. - #[inline] - #[cfg_attr(not(test), expect(dead_code, reason = "test-only catalog"))] - pub(crate) fn catalog(&self) -> &Catalog { - self.inner().catalog() - } - - /// Return the shared logical lock manager. - #[inline] - #[cfg_attr(not(test), expect(dead_code, reason = "test-only lock_manager"))] - pub(crate) fn lock_manager(&self) -> &QuiescentGuard { - self.inner().lock_manager() - } - - /// Try to clone the crate-private shared runtime handle while the engine is - /// still running. - #[cfg(test)] - #[inline] - pub(crate) fn new_ref(&self) -> LifecycleResult { - let inner = self.inner(); - inner.with_admitted_operation(|| EngineRef::new(Arc::clone(inner))) - } - /// Try to complete idempotent engine shutdown without waiting for active work. /// /// `try_shutdown` rejects new work immediately, drains in-flight admission, @@ -382,12 +368,11 @@ impl Engine { let observer_count = blocker .as_ref() .map_or(0, |blocker| blocker.observer_count()); - let cleanup_queued = self.queue_shutdown_operation_cleanup( - inner, - blocker.as_ref().and_then(|blocker| blocker.cleanup()), - ); + let has_session_blocker = blocker.is_some(); + let cleanup_queued = self + .queue_shutdown_operation_cleanup(blocker.and_then(|blocker| blocker.into_cleanup())); let (mandatory_callers, mandatory_internal) = inner.mandatory_runtime.blocker_counts(); - if blocker.is_some() || mandatory_callers != 0 || mandatory_internal != 0 { + if has_session_blocker || mandatory_callers != 0 || mandatory_internal != 0 { obs::warn!( "event=engine_lifecycle component=engine action=shutdown_finish result=busy mode=try origin=explicit session_blocker={} operation_state={} observer_count={} cleanup_queued={} mandatory_callers={} mandatory_internal={}", session_blocker, @@ -456,7 +441,7 @@ impl Engine { drop(_shutdown); if let Some(shutdown_wait) = shutdown_wait { - self.queue_shutdown_operation_cleanup(inner, shutdown_wait.blocker.cleanup()); + self.queue_shutdown_operation_cleanup(shutdown_wait.blocker.into_cleanup()); shutdown_wait.listener.wait(); } } @@ -481,18 +466,17 @@ impl Engine { /// active operation states only block shutdown; accepted mandatory work /// already owns its cleanup authority through the stable operation entry. #[inline] - fn queue_shutdown_operation_cleanup( - &self, - inner: &Arc, - cleanup: Option<(SessionOperationKey, TrxID)>, - ) -> bool { - let Some((operation_key, trx_id)) = cleanup else { + fn queue_shutdown_operation_cleanup(&self, cleanup: Option) -> bool { + let Some(SessionCleanupRequest { + runtime, + operation_key, + trx_id, + }) = cleanup + else { return false; }; - let engine_ref = EngineRef::new(Arc::clone(inner)); - inner - .trx_sys - .request_abandoned_trx_cleanup(engine_ref, operation_key, trx_id); + let trx_sys = runtime.trx_sys.clone(); + trx_sys.request_abandoned_trx_cleanup(runtime, operation_key, trx_id); true } } @@ -510,118 +494,12 @@ impl Drop for Engine { } } -/// Crate-private cloneable shared runtime handle for the storage engine. +/// Immutable component capabilities retained by registered session state. /// -/// `EngineRef` intentionally does not own shutdown orchestration. It exposes -/// shared memory reachability and component access to admitted session work, -/// mandatory work, and bounded cleanup sections. Its clone/drop lifecycle is -/// not itself an authoritative shutdown blocker. -#[derive(Clone)] -pub(crate) struct EngineRef(Arc); - -impl EngineRef { - #[inline] - fn new(inner: Arc) -> Self { - Self(inner) - } - - /// Downgrade this private runtime handle into weak engine reachability. - #[inline] - pub(crate) fn downgrade(&self) -> WeakEngineRef { - WeakEngineRef(Arc::downgrade(&self.0)) - } - - /// Create a new session while the engine is still running. - #[inline] - #[cfg_attr( - not(test), - expect(dead_code, reason = "transitional internal runtime handle") - )] - pub(crate) fn new_session(&self) -> LifecycleResult { - self.0.with_admitted_operation(|| { - let id = self.0.next_session_id(); - self.0 - .session_registry - .create_session(&self.0, self.clone(), id) - }) - } - - /// Return the shared catalog handle. - #[inline] - pub(crate) fn catalog(&self) -> &Catalog { - &self.0.catalog - } - - /// Clone the catalog component guard for transferable runtime authority. - #[inline] - pub(crate) fn catalog_guard(&self) -> QuiescentGuard { - self.0.catalog.clone() - } - - /// Return the shared logical lock manager. - #[inline] - pub(crate) fn lock_manager(&self) -> &QuiescentGuard { - self.0.lock_manager() - } - - /// Returns the next engine-local session identity. - #[inline] - #[cfg_attr(not(test), expect(dead_code, reason = "pending dead-code audit"))] - pub(crate) fn next_session_id(&self) -> SessionID { - self.0.next_session_id() - } -} - -impl Deref for EngineRef { - type Target = EngineInner; - - #[inline] - fn deref(&self) -> &EngineInner { - &self.0 - } -} - -/// Crate-private weak reachability handle used by public runtime handles. -#[derive(Clone)] -pub(crate) struct WeakEngineRef(Weak); - -impl WeakEngineRef { - /// Create weak engine reachability from the engine runtime owner. - #[inline] - pub(crate) fn new(inner: &Arc) -> Self { - Self(Arc::downgrade(inner)) - } - - /// Upgrade weak engine reachability for one admitted public operation. - #[inline] - pub(crate) fn upgrade(&self) -> LifecycleResult { - self.0.upgrade().map(EngineRef::new).ok_or_else(|| { - Report::new(LifecycleError::Shutdown).attach("engine is no longer reachable") - }) - } - - /// Upgrade weak reachability for explicit terminal cleanup. - /// - /// This path does not acquire foreground admission: an already-active - /// transaction must be able to commit or roll back while owner shutdown is - /// waiting for active transactions to finish before component teardown. - #[inline] - pub(crate) fn upgrade_for_terminal(&self) -> LifecycleResult { - self.upgrade() - } - - /// Best-effort upgrade for nonblocking cleanup hints from `Drop`. - #[inline] - pub(crate) fn upgrade_for_cleanup(&self) -> Option { - self.0.upgrade().map(EngineRef::new) - } -} - -/// Shared crate-private runtime state for an [`Engine`]. -/// -/// The fields here are the cloneable handles that sessions and other runtime -/// objects may retain. Owner-only teardown state lives on [`Engine`] itself. -pub(crate) struct EngineInner { +/// The weak registry edge is used only for pointer-exact removal after a +/// session becomes closed and idle. It must never be used for operation +/// resolution. +pub(crate) struct EngineCore { /// Engine-level fatal runtime poison state. pub(crate) poisoner: QuiescentGuard, /// Engine-owned scheduler for accepted caller and internal obligations. @@ -630,22 +508,14 @@ pub(crate) struct EngineInner { pub(crate) catalog: QuiescentGuard, /// Shared transaction-system handle. pub(crate) trx_sys: QuiescentGuard, - /// Metadata pool used for block-index and catalog tables. - pub(crate) meta_pool: MetaPool, - /// Secondary-index pool. - pub(crate) index_pool: IndexPool, - /// In-memory row-page pool for table data. - pub(crate) mem_pool: MemPool, + /// Canonical typed pool handles and matching guard bundle. + pub(crate) pools: EnginePools, /// Table-file subsystem that runs persistent page IO. pub(crate) table_fs: QuiescentGuard, - /// Global readonly pool for persisted table-file reads. - pub(crate) disk_pool: DiskPool, /// Shared logical metadata and table-data lock manager. lock_manager: QuiescentGuard, - /// Engine-owned strong session-state registry. - pub(crate) session_registry: SessionRegistry, - /// Monotonically increasing engine-local session identity source. - next_session_id: AtomicU64, + /// Cold weak back-reference for pointer-exact idle-session removal. + pub(crate) session_registry: Weak, /// Per-engine table-DDL fault and phase controller. #[cfg(test)] pub(crate) table_ddl_test: TableDdlTestController, @@ -655,25 +525,19 @@ pub(crate) struct EngineInner { /// Per-engine maintenance fault and phase controller. #[cfg(test)] pub(crate) maintenance_test: MaintenanceTestController, - lifecycle: EngineLifecycle, } -impl EngineInner { +impl EngineCore { /// Return the shared catalog handle. #[inline] pub(crate) fn catalog(&self) -> &Catalog { &self.catalog } - /// Clone the inner engine buffer-pool handles as one startup/recovery bundle. + /// Borrow the canonical pool guard bundle. #[inline] - pub(crate) fn pools(&self) -> EnginePools { - EnginePools::new( - self.meta_pool.clone_inner(), - self.index_pool.clone_inner(), - self.mem_pool.clone_inner(), - self.disk_pool.clone_inner(), - ) + pub(crate) fn pool_guards(&self) -> &PoolGuards { + self.pools.pool_guards() } /// Return the shared logical lock manager. @@ -681,7 +545,24 @@ impl EngineInner { pub(crate) fn lock_manager(&self) -> &QuiescentGuard { &self.lock_manager } +} +/// Owner-facing coordination shell for one [`Engine`]. +/// +/// Registered sessions retain only [`EngineCore`] and the lifecycle admission +/// gate, so no session-local authority can recover this owner shell. +pub(crate) struct EngineInner { + /// Shared component capabilities. + pub(crate) core: Arc, + /// Engine-owned strong session-state registry. + pub(crate) session_registry: Arc, + /// Shared lifecycle admission and shutdown state. + lifecycle: Arc, + /// Monotonically increasing engine-local session identity source. + next_session_id: AtomicU64, +} + +impl EngineInner { /// Returns the next engine-local session identity. #[inline] pub(crate) fn next_session_id(&self) -> SessionID { @@ -706,31 +587,6 @@ impl EngineInner { Ok(admission) } - /// Enter one poison-tolerant inspection while lifecycle admission is open. - /// - /// The returned token closes inspection registration against shutdown but - /// deliberately does not validate storage health. Callers must restrict - /// the admitted work to read-only diagnostics and register its session - /// observer before releasing the token. - #[inline] - pub(crate) fn acquire_inspection_admission(&self) -> LifecycleResult> { - self.lifecycle - .admit() - .attach_with(|| "phase=acquire_engine_inspection_admission") - } - - /// Returns whether owner-side shutdown has started. - #[inline] - pub(crate) fn shutdown_started(&self) -> bool { - self.lifecycle.inspect_state() != EngineLifecycleState::Running - } - - /// Registers for owner-side shutdown start. - #[inline] - pub(crate) fn shutdown_listener(&self) -> EventListener { - self.lifecycle.shutdown_listener() - } - /// Run immediate synchronous work under engine admission. /// /// Use this helper for lifecycle validation plus local runtime lookup or @@ -743,6 +599,15 @@ impl EngineInner { } } +impl Deref for EngineInner { + type Target = EngineCore; + + #[inline] + fn deref(&self) -> &Self::Target { + &self.core + } +} + #[inline] async fn bootstrap_inner(config: EngineConfig) -> Result { let resolved = config @@ -881,26 +746,34 @@ async fn bootstrap_inner(config: EngineConfig) -> Result { let table_fs = registry.dependency::(); let disk_pool = registry.dependency::(); let lock_manager = registry.dependency::(); - let engine_inner = EngineInner { + let session_registry = Arc::new(SessionRegistry::new()); + let lifecycle = Arc::new(EngineLifecycle::new()); + let core = Arc::new(EngineCore { poisoner, mandatory_runtime, catalog, trx_sys, - meta_pool, - index_pool, - mem_pool, + pools: EnginePools::new( + meta_pool.clone_inner(), + index_pool.clone_inner(), + mem_pool.clone_inner(), + disk_pool.clone_inner(), + ), table_fs, - disk_pool, lock_manager, - session_registry: SessionRegistry::new(), - next_session_id: AtomicU64::new(FIRST_SESSION_ID.as_u64()), + session_registry: Arc::downgrade(&session_registry), #[cfg(test)] table_ddl_test: TableDdlTestController::default(), #[cfg(test)] index_ddl_test: IndexDdlTestController::default(), #[cfg(test)] maintenance_test: MaintenanceTestController::default(), - lifecycle: EngineLifecycle::new(), + }); + let engine_inner = EngineInner { + core, + session_registry, + lifecycle, + next_session_id: AtomicU64::new(FIRST_SESSION_ID.as_u64()), }; Ok(Engine { inner: Arc::new(engine_inner), @@ -915,8 +788,8 @@ mod tests { use crate::catalog::tests::table1; use crate::conf::{EngineConfig, EvictableBufferPoolConfig, FileSystemConfig, TrxSysConfig}; use crate::error::{ - ConfigError, DiscloseError, Error, ErrorKind, FatalError, LifecycleError, OperationError, - ResourceError, RuntimeError, + ConfigError, Error, ErrorKind, FatalError, LifecycleError, OperationError, ResourceError, + RuntimeError, }; use crate::file::fs::tests::io_backend_stats_handle_identity as fs_stats_handle_identity; use crate::id::{OperationID, SessionOperationKey, TableID, TrxID}; @@ -1271,7 +1144,7 @@ mod tests { } fn lock_entry_count(engine: &Engine, owner: LockOwner) -> usize { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .filter(|entry| entry.owner == owner) @@ -1312,16 +1185,14 @@ mod tests { } #[test] - fn test_session_ids_are_monotonic_across_engine_handles() { + fn test_session_ids_are_monotonic_across_engine_sessions() { smol::block_on(async { let root = TempDir::new().unwrap(); let engine = Engine::bootstrap(test_engine_config_for(root.path())) .await .unwrap(); - let engine_ref = engine.new_ref().unwrap(); - let session1 = engine.new_session().unwrap(); - let session2 = engine_ref.new_session().unwrap(); + let session2 = engine.new_session().unwrap(); let session3 = engine.new_session().unwrap(); assert_eq!(session1.id(), FIRST_SESSION_ID); @@ -1337,26 +1208,33 @@ mod tests { let engine = Engine::bootstrap(test_engine_config_for(root.path())) .await .unwrap(); - let engine_ref = engine.new_ref().unwrap(); + let session = engine.new_session().unwrap(); + let runtime = session.engine(); let resource = LockResource::TableMetadata(TableID::new(10)); let owner = LockOwner::session_explicit(SessionID::new(10)); assert!( - try_acquire(engine.lock_manager(), resource, LockMode::Exclusive, owner).unwrap() + try_acquire( + engine.inner().core.lock_manager(), + resource, + LockMode::Exclusive, + owner + ) + .unwrap() ); assert!( !try_acquire( - engine_ref.lock_manager(), + runtime.lock_manager(), resource, LockMode::Shared, LockOwner::session_explicit(SessionID::new(11)) ) .unwrap() ); - assert_eq!(engine_ref.lock_manager().release_owner(owner), 1); + assert_eq!(runtime.lock_manager().release_owner(owner), 1); assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), resource, LockMode::Shared, LockOwner::session_explicit(SessionID::new(11)) @@ -1378,7 +1256,7 @@ mod tests { assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), resource, LockMode::Exclusive, LockOwner::session_explicit(session.id()) @@ -1389,7 +1267,7 @@ mod tests { assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), resource, LockMode::Shared, LockOwner::session_explicit(SessionID::new(91_201)) @@ -1417,7 +1295,7 @@ mod tests { assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), resource, LockMode::Exclusive, explicit_owner, @@ -1426,7 +1304,7 @@ mod tests { ); assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), resource, LockMode::IntentShared, maintenance_owner, @@ -1436,18 +1314,22 @@ mod tests { drop(session); - assert!(!engine.lock_manager().owner_holds( + assert!(!engine.inner().core.lock_manager().owner_holds( resource, explicit_owner, LockMode::IntentShared, )); - assert!(engine.lock_manager().owner_holds( + assert!(engine.inner().core.lock_manager().owner_holds( resource, maintenance_owner, LockMode::IntentShared, )); assert_eq!( - engine.lock_manager().release(resource, maintenance_owner), + engine + .inner() + .core + .lock_manager() + .release(resource, maintenance_owner), 1 ); }); @@ -1464,7 +1346,7 @@ mod tests { let blocking_owner = LockOwner::session_explicit(SessionID::new(91_203)); assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), resource, LockMode::Exclusive, blocking_owner @@ -1474,7 +1356,7 @@ mod tests { let session = engine.new_session().unwrap(); let waiting_owner = LockOwner::session_explicit(session.id()); - let manager = engine.lock_manager().clone(); + let manager = engine.inner().core.lock_manager().clone(); let wait_task = smol::spawn(async move { manager .acquire(resource, LockMode::Shared, waiting_owner) @@ -1483,7 +1365,7 @@ mod tests { let mut waiter_seen = false; for _ in 0..100 { - waiter_seen = debug_snapshot(engine.lock_manager()) + waiter_seen = debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .any(|entry| entry.owner == waiting_owner); @@ -1497,7 +1379,14 @@ mod tests { drop(session); let err = wait_task.await.unwrap_err(); assert_eq!(*err.current_context(), OperationError::LockWaiterReleased); - assert_eq!(engine.lock_manager().release_owner(blocking_owner), 1); + assert_eq!( + engine + .inner() + .core + .lock_manager() + .release_owner(blocking_owner), + 1 + ); }); } @@ -1523,8 +1412,8 @@ mod tests { .unwrap(); let table_stats = fs_stats_handle_identity(&engine.inner().table_fs); - let mem_stats = pool_stats_handle_identity(&engine.inner().mem_pool); - let index_stats = pool_stats_handle_identity(&engine.inner().index_pool); + let mem_stats = pool_stats_handle_identity(&engine.inner().pools.mem); + let index_stats = pool_stats_handle_identity(&engine.inner().pools.index); assert_eq!(table_stats, mem_stats); assert_eq!(table_stats, index_stats); @@ -1554,11 +1443,11 @@ mod tests { assert_eq!(engine.inner().table_fs.configured_io_depth(), 7); assert_eq!( - engine.inner().mem_pool.io_backend_stats(), + engine.inner().pools.mem.io_backend_stats(), engine.inner().table_fs.io_backend_stats() ); assert_eq!( - engine.inner().index_pool.io_backend_stats(), + engine.inner().pools.index.io_backend_stats(), engine.inner().table_fs.io_backend_stats() ); }); @@ -1868,37 +1757,6 @@ mod tests { Err(err) => err, }; assert_runtime_unavailable_after_shutdown(err); - - let err = match engine.new_ref() { - Ok(_) => panic!("expected shutdown error"), - Err(err) => err.disclose(), - }; - assert_runtime_unavailable_after_shutdown(err); - }); - } - - #[test] - fn test_engine_ref_does_not_block_shutdown_or_bypass_closed_admission() { - smol::block_on(async { - let root = TempDir::new().unwrap(); - let engine = Engine::bootstrap(test_engine_config_for(root.path())) - .await - .unwrap(); - let engine_ref = engine.new_ref().unwrap(); - - engine.try_shutdown().unwrap(); - - let err = match engine_ref.new_session() { - Ok(_) => panic!("expected shutdown error"), - Err(err) => err.disclose(), - }; - assert_runtime_unavailable_after_shutdown(err); - - let err = match engine.new_ref() { - Ok(_) => panic!("expected shutdown error"), - Err(err) => err.disclose(), - }; - assert_runtime_unavailable_after_shutdown(err); }); } @@ -2599,7 +2457,7 @@ mod tests { smol::block_on(Engine::bootstrap(test_engine_config_for(root.path()))).unwrap(); let mut session = engine.new_session().unwrap(); let trx = session.begin_trx().unwrap(); - let shutdown_started = engine.inner().shutdown_listener(); + let shutdown_started = engine.inner().lifecycle.shutdown_listener(); let (done_tx, done_rx) = mpsc::channel(); thread::scope(|scope| { @@ -2648,7 +2506,7 @@ mod tests { config, engine.inner().poisoner.clone(), engine.inner().mandatory_runtime.clone(), - engine.inner().pools(), + engine.inner().core.pools.clone(), engine.inner().table_fs.clone(), engine.inner().catalog.clone(), ) diff --git a/doradb-storage/src/index/mod.rs b/doradb-storage/src/index/mod.rs index 91e77e13..1f7b5012 100644 --- a/doradb-storage/src/index/mod.rs +++ b/doradb-storage/src/index/mod.rs @@ -13,7 +13,7 @@ mod secondary_index; mod unique_index; pub(crate) mod util; -use crate::buffer::{BufferPool, PoolGuards}; +use crate::buffer::{BufferPool, PoolGuard, PoolGuards}; use crate::error::RuntimeResult; use crate::id::BlockID; use crate::table::TableRootSnapshot; @@ -136,7 +136,8 @@ impl<'op, 'idx, P: BufferPool + 'static> CurrentIndexReadHandle<'op, 'idx, P> { /// Owned executable state retained by one caller-driven index stream. pub(crate) struct OwnedCurrentIndexReadHandle<'trx, P: BufferPool + 'static> { index: Arc>, - guards: PoolGuards, + index_pool_guard: PoolGuard, + disk_pool_guard: PoolGuard, root: BlockID, _transaction: PhantomData<&'trx mut Transaction>, } @@ -146,14 +147,16 @@ impl<'trx, P: BufferPool + 'static> OwnedCurrentIndexReadHandle<'trx, P> { #[inline] pub(crate) fn new( index: Arc>, - guards: PoolGuards, + index_pool_guard: PoolGuard, + disk_pool_guard: PoolGuard, root: BlockID, _proof: &TrxReadProof<'_>, _transaction: PhantomData<&'trx mut Transaction>, ) -> Self { Self { index, - guards, + index_pool_guard, + disk_pool_guard, root, _transaction: PhantomData, } diff --git a/doradb-storage/src/index/owned_stream.rs b/doradb-storage/src/index/owned_stream.rs index 43f514be..95681318 100644 --- a/doradb-storage/src/index/owned_stream.rs +++ b/doradb-storage/src/index/owned_stream.rs @@ -266,7 +266,8 @@ impl<'trx, P: BufferPool + 'static> OwnedSecondaryIndexCandidateStream<'trx, P> pub(crate) fn new(handle: OwnedCurrentIndexReadHandle<'trx, P>, range: KeyRange) -> Self { let OwnedCurrentIndexReadHandle { index, - guards: pool_guards, + index_pool_guard, + disk_pool_guard, root, _transaction, } = handle; @@ -274,14 +275,11 @@ impl<'trx, P: BufferPool + 'static> OwnedSecondaryIndexCandidateStream<'trx, P> let inner = match index.as_ref() { SecondaryIndex::Unique { .. } => { let mem = OwnedUniqueMemIndexCandidateStream::new( - OwnedUniqueMemIndexCursor::new( - Arc::clone(&index), - pool_guards.index_guard().clone(), - ), + OwnedUniqueMemIndexCursor::new(Arc::clone(&index), index_pool_guard), Arc::clone(&range), ); let disk = OwnedUniqueDiskTreeCandidateStream::new( - OwnedUniqueDiskTreeCursor::new(index, pool_guards.disk_guard().clone(), root), + OwnedUniqueDiskTreeCursor::new(index, disk_pool_guard, root), range, ); OwnedSecondaryIndexCandidateStreamKind::Unique(SecondaryIndexCandidateStream::new( @@ -290,18 +288,11 @@ impl<'trx, P: BufferPool + 'static> OwnedSecondaryIndexCandidateStream<'trx, P> } SecondaryIndex::NonUnique { .. } => { let mem = OwnedNonUniqueMemIndexCandidateStream::new( - OwnedNonUniqueMemIndexCursor::new( - Arc::clone(&index), - pool_guards.index_guard().clone(), - ), + OwnedNonUniqueMemIndexCursor::new(Arc::clone(&index), index_pool_guard), Arc::clone(&range), ); let disk = OwnedNonUniqueDiskTreeCandidateStream::new( - OwnedNonUniqueDiskTreeCursor::new( - index, - pool_guards.disk_guard().clone(), - root, - ), + OwnedNonUniqueDiskTreeCursor::new(index, disk_pool_guard, root), range, ); OwnedSecondaryIndexCandidateStreamKind::NonUnique( diff --git a/doradb-storage/src/index/row_page_index.rs b/doradb-storage/src/index/row_page_index.rs index ae87b845..2ffbfea7 100644 --- a/doradb-storage/src/index/row_page_index.rs +++ b/doradb-storage/src/index/row_page_index.rs @@ -2055,19 +2055,19 @@ mod tests { .unwrap(); { let metadata = make_test_metadata(); - let meta_guard = engine.inner().meta_pool.pool_guard(); + let meta_guard = engine.inner().pools.meta.pool_guard(); let blk_idx = RowPageIndex::new( - engine.inner().meta_pool.clone_inner(), + engine.inner().pools.meta.clone(), &meta_guard, RowID::new(0), ) .await .expect("test row-page-index construction should succeed"); - let mem_guard = engine.inner().mem_pool.pool_guard(); + let mem_guard = engine.inner().pools.mem.pool_guard(); let p1 = blk_idx .get_insert_page( &meta_guard, - &*engine.inner().mem_pool, + &*engine.inner().pools.mem, &mem_guard, &metadata.col, 100, @@ -2081,7 +2081,7 @@ mod tests { let p2 = blk_idx .get_insert_page( &meta_guard, - &*engine.inner().mem_pool, + &*engine.inner().pools.mem, &mem_guard, &metadata.col, 100, @@ -2115,19 +2115,19 @@ mod tests { .unwrap(); { let metadata = make_test_metadata(); - let meta_guard = engine.inner().meta_pool.pool_guard(); + let meta_guard = engine.inner().pools.meta.pool_guard(); let blk_idx = RowPageIndex::new( - engine.inner().meta_pool.clone_inner(), + engine.inner().pools.meta.clone(), &meta_guard, RowID::new(0), ) .await .expect("test row-page-index construction should succeed"); - let mem_guard = engine.inner().mem_pool.pool_guard(); + let mem_guard = engine.inner().pools.mem.pool_guard(); let p1 = blk_idx .get_insert_page_exclusive( &meta_guard, - &*engine.inner().mem_pool, + &*engine.inner().pools.mem, &mem_guard, &metadata.col, 100, @@ -2140,7 +2140,7 @@ mod tests { let p2 = blk_idx .get_insert_page_exclusive( &meta_guard, - &*engine.inner().mem_pool, + &*engine.inner().pools.mem, &mem_guard, &metadata.col, 100, @@ -2497,20 +2497,20 @@ mod tests { .unwrap(); { let metadata = make_test_metadata(); - let meta_guard = engine.inner().meta_pool.pool_guard(); + let meta_guard = engine.inner().pools.meta.pool_guard(); let blk_idx = RowPageIndex::new( - engine.inner().meta_pool.clone_inner(), + engine.inner().pools.meta.clone(), &meta_guard, RowID::new(0), ) .await .expect("test row-page-index construction should succeed"); - let mem_guard = engine.inner().mem_pool.pool_guard(); + let mem_guard = engine.inner().pools.mem.pool_guard(); for _ in 0..row_pages { let _ = blk_idx .get_insert_page( &meta_guard, - &*engine.inner().mem_pool, + &*engine.inner().pools.mem, &mem_guard, &metadata.col, 100, @@ -2939,21 +2939,21 @@ mod tests { .unwrap(); { let metadata = make_test_metadata(); - let meta_guard = engine.inner().meta_pool.pool_guard(); + let meta_guard = engine.inner().pools.meta.pool_guard(); let blk_idx = RowPageIndex::new( - engine.inner().meta_pool.clone_inner(), + engine.inner().pools.meta.clone(), &meta_guard, RowID::new(0), ) .await .expect("test row-page-index construction should succeed"); - let mem_guard = engine.inner().mem_pool.pool_guard(); + let mem_guard = engine.inner().pools.mem.pool_guard(); let redo_ctx = RowPageCreateRedoCtx::new(&engine.inner().trx_sys, TableID::new(104)); let page_guard = blk_idx .get_insert_page_with_redo( &meta_guard, - &*engine.inner().mem_pool, + &*engine.inner().pools.mem, &mem_guard, &metadata.col, 100, @@ -2971,7 +2971,7 @@ mod tests { let reused_page = blk_idx .get_insert_page_with_redo( &meta_guard, - &*engine.inner().mem_pool, + &*engine.inner().pools.mem, &mem_guard, &metadata.col, 100, @@ -3022,15 +3022,15 @@ mod tests { .await .unwrap(); let metadata = make_test_metadata(); - let meta_guard = engine.inner().meta_pool.pool_guard(); + let meta_guard = engine.inner().pools.meta.pool_guard(); let blk_idx = RowPageIndex::new( - engine.inner().meta_pool.clone_inner(), + engine.inner().pools.meta.clone(), &meta_guard, RowID::new(0), ) .await .expect("test row-page-index construction should succeed"); - let mem_guard = engine.inner().mem_pool.pool_guard(); + let mem_guard = engine.inner().pools.mem.pool_guard(); let redo_ctx = RowPageCreateRedoCtx::new(&engine.inner().trx_sys, TableID::new(206)); let _ = engine .inner() @@ -3040,7 +3040,7 @@ mod tests { let err = match blk_idx .get_insert_page_with_redo( &meta_guard, - &*engine.inner().mem_pool, + &*engine.inner().pools.mem, &mem_guard, &metadata.col, 100, @@ -3068,7 +3068,8 @@ mod tests { }; let page = engine .inner() - .mem_pool + .pools + .mem .get_page::(&mem_guard, page_id, LatchFallbackMode::Shared) .await .expect("published row page should remain allocated") @@ -3095,12 +3096,11 @@ mod tests { .await .unwrap(); { - let meta_pool = &engine.inner().meta_pool; + let meta_pool = &engine.inner().pools.meta; let meta_guard = meta_pool.pool_guard(); - let blk_idx = - RowPageIndex::new(meta_pool.clone_inner(), &meta_guard, RowID::new(0)) - .await - .expect("test row-page-index construction should succeed"); + let blk_idx = RowPageIndex::new(meta_pool.clone(), &meta_guard, RowID::new(0)) + .await + .expect("test row-page-index construction should succeed"); let redo_ctx = RowPageCreateRedoCtx::new(&engine.inner().trx_sys, table_id); let total_pages = NBR_ROW_PAGE_ENTRIES_IN_LEAF + 64; let worker_count = 8usize; diff --git a/doradb-storage/src/log/mod.rs b/doradb-storage/src/log/mod.rs index 14ccb922..74270288 100644 --- a/doradb-storage/src/log/mod.rs +++ b/doradb-storage/src/log/mod.rs @@ -2440,7 +2440,7 @@ mod tests { use crate::buffer::{PoolRole, test_page_id}; use crate::catalog::tests::table2; use crate::conf::{EngineConfig, EvictableBufferPoolConfig, TrxSysConfig}; - use crate::engine::{Engine, EngineRef}; + use crate::engine::Engine; use crate::error::{ DataIntegrityError, ErrorKind, FatalError, IoError, IoResult, LifecycleError, Result, RuntimeError, SharedFatalError, @@ -2856,10 +2856,13 @@ mod tests { } } - fn spawn_sys_commit_wait(engine: EngineRef, marker: u64) -> JoinHandle> { + fn spawn_sys_commit_wait( + trx_sys: QuiescentGuard, + marker: u64, + ) -> JoinHandle> { thread::spawn(move || { smol::block_on(async move { - let mut sys_trx = engine.trx_sys.begin_sys_trx(); + let mut sys_trx = trx_sys.begin_sys_trx(); sys_trx.create_row_page( TableID::from(marker), PageID::from(marker), @@ -2867,7 +2870,7 @@ mod tests { RowID::new(1), ); let prepared = sys_trx.prepare(); - engine.trx_sys.commit_prepared(prepared).await + trx_sys.commit_prepared(prepared).await }) }) } @@ -3419,7 +3422,6 @@ mod tests { redo_bin: None, payload: None, attachment: None, - lock_manager: None, lock_state: None, trx_inner: None, }], @@ -3936,7 +3938,6 @@ mod tests { redo_bin: None, payload: None, attachment: None, - lock_manager: None, lock_state: None, trx_inner: None, }], @@ -4033,7 +4034,6 @@ mod tests { redo_bin: None, payload: None, attachment: None, - lock_manager: None, lock_state: None, trx_inner: None, }], @@ -5286,10 +5286,10 @@ mod tests { let hook = ControlledRedoWriteHook::new(redo_fd, libc::EIO); let _install = install_storage_backend_test_hook(Arc::new(hook.clone())); - let commit1 = spawn_sys_commit_wait(engine.new_ref().unwrap(), 1); + let commit1 = spawn_sys_commit_wait(engine.inner().core.trx_sys.clone(), 1); hook.wait_started(1).await; - let commit2 = spawn_sys_commit_wait(engine.new_ref().unwrap(), 2); + let commit2 = spawn_sys_commit_wait(engine.inner().core.trx_sys.clone(), 2); wait_for(|| { !engine .inner() @@ -5339,10 +5339,10 @@ mod tests { let hook = ControlledRedoSyncHook::new(redo_fd, sync_kind, libc::EIO); let _install = install_storage_backend_test_hook(Arc::new(hook.clone())); - let commit1 = spawn_sys_commit_wait(engine.new_ref().unwrap(), 10); + let commit1 = spawn_sys_commit_wait(engine.inner().core.trx_sys.clone(), 10); hook.wait_started(1).await; - let commit2 = spawn_sys_commit_wait(engine.new_ref().unwrap(), 11); + let commit2 = spawn_sys_commit_wait(engine.inner().core.trx_sys.clone(), 11); wait_for(|| { !engine .inner() diff --git a/doradb-storage/src/log/prefix.rs b/doradb-storage/src/log/prefix.rs index f6059343..75ef8b32 100644 --- a/doradb-storage/src/log/prefix.rs +++ b/doradb-storage/src/log/prefix.rs @@ -427,7 +427,6 @@ mod tests { redo_bin: None, payload: None, attachment: None, - lock_manager: None, lock_state: None, trx_inner: None, }], diff --git a/doradb-storage/src/recovery/mod.rs b/doradb-storage/src/recovery/mod.rs index 36e78e2f..1bf4d754 100644 --- a/doradb-storage/src/recovery/mod.rs +++ b/doradb-storage/src/recovery/mod.rs @@ -1472,7 +1472,13 @@ mod tests { "test setup should create retained redo suffix" ); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); assert_checkpoint_published(&mut session, table.table_id()).await; drop(table); let mut durability_trx = session.begin_trx().unwrap(); @@ -1491,7 +1497,7 @@ mod tests { .checkpoint_catalog() .await .unwrap(); - publish_first_redo_log_seq_for_test(&engine.catalog().storage, 1) + publish_first_redo_log_seq_for_test(&engine.inner().core.catalog().storage, 1) .await .unwrap(); (engine, table_id) @@ -1512,6 +1518,8 @@ mod tests { .await .unwrap(); let replay_floor = engine + .inner() + .core .catalog() .storage .checkpoint_snapshot() @@ -1620,9 +1628,9 @@ mod tests { catalog_replay_start_ts: TrxID, ) -> RecoveryCoordinator<'a> { let resources = RecoveryResources::new( - engine.inner().pools(), + engine.inner().core.pools.clone(), engine.inner().table_fs.clone(), - engine.catalog(), + engine.inner().core.catalog(), ); let config = &engine.inner().trx_sys.config; let file_prefix = config.file_prefix().unwrap(); @@ -1795,6 +1803,8 @@ mod tests { trx.exec(async |stmt| { assert!( engine + .inner() + .core .catalog() .storage .tables() @@ -1803,6 +1813,8 @@ mod tests { .disclose()? ); engine + .inner() + .core .catalog() .storage .tables() @@ -1816,6 +1828,8 @@ mod tests { .await .disclose()?; engine + .inner() + .core .catalog() .storage .indexes() @@ -1830,6 +1844,8 @@ mod tests { .await .disclose()?; engine + .inner() + .core .catalog() .storage .index_columns() @@ -1865,6 +1881,8 @@ mod tests { trx.exec(async |stmt| { assert_eq!( engine + .inner() + .core .catalog() .storage .index_columns() @@ -1875,6 +1893,8 @@ mod tests { ); assert!( engine + .inner() + .core .catalog() .storage .indexes() @@ -1902,7 +1922,13 @@ mod tests { metadata: Arc, cts: TrxID, ) { - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let table_file = Arc::clone(table.file()); let mut roots = table_file .active_root_unchecked() @@ -1935,13 +1961,21 @@ mod tests { next_index_no: u16, index_one_active: bool, ) { - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let metadata = table.metadata(); assert_eq!(metadata.idx.next_index_no(), next_index_no); assert_eq!(metadata.idx.index_spec(1).is_some(), index_one_active); let session = engine.new_session().unwrap(); let table_obj = engine + .inner() + .core .catalog() .storage .tables() @@ -1951,6 +1985,8 @@ mod tests { .unwrap(); assert_eq!(table_obj.next_index_no, next_index_no); let indexes = engine + .inner() + .core .catalog() .storage .indexes() @@ -2384,7 +2420,7 @@ mod tests { .checkpoint_catalog() .await .unwrap(); - let snapshot = engine.catalog().storage.checkpoint_snapshot(); + let snapshot = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert!(snapshot.catalog_replay_start_ts > ddl_cts); drop(engine); @@ -2427,7 +2463,7 @@ mod tests { .checkpoint_catalog() .await .unwrap(); - let snapshot = engine.catalog().storage.checkpoint_snapshot(); + let snapshot = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert!(snapshot.catalog_replay_start_ts > ddl_cts); drop(engine); @@ -2637,6 +2673,8 @@ mod tests { drop(session); let batch = engine + .inner() + .core .catalog() .scan_checkpoint_batch( engine.inner().trx_sys.persisted_watermark_cts(), @@ -2741,8 +2779,22 @@ mod tests { .await .unwrap(); - assert!(engine.catalog().get_table(table_id).await.is_some()); - let table = engine.catalog().get_table(table_id).await.unwrap(); + assert!( + engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_some() + ); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); assert_eq!(table.metadata().as_ref(), &expected_metadata); drop(table); @@ -2843,7 +2895,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let session = engine.new_session().unwrap(); let mut rows = 0usize; { @@ -2903,7 +2961,7 @@ mod tests { .checkpoint_catalog() .await .unwrap(); - let snap = engine.catalog().storage.checkpoint_snapshot(); + let snap = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert!(snap.catalog_replay_start_ts > MIN_SNAPSHOT_TS); drop(session); @@ -2923,7 +2981,15 @@ mod tests { .await .unwrap(); - assert!(engine.catalog().get_table(table_id).await.is_some()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_some() + ); drop(engine); }) } @@ -2941,11 +3007,19 @@ mod tests { .unwrap(); let table_id = create_index_ddl_base_table(&engine, vec![base_unique_index_spec()]).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let root_floor = table.redo_replay_floor_snapshot(); let mut session = engine.new_session().unwrap(); assert_checkpoint_published(&mut session, table.table_id()).await; let watermark = engine + .inner() + .core .catalog() .storage .table_replay_silent_watermarks() @@ -2972,6 +3046,8 @@ mod tests { session.checkpoint_catalog().await.unwrap(); assert_eq!( engine + .inner() + .core .catalog() .storage .checkpointed_silent_watermarks() @@ -3002,14 +3078,23 @@ mod tests { )) .await .unwrap(); - let snapshot = recovered.catalog().storage.checkpoint_snapshot(); + let snapshot = recovered + .inner() + .core + .catalog() + .storage + .checkpoint_snapshot(); let (live_before_catalog_checkpoint, _) = recovered + .inner() + .core .catalog() .snapshot_user_table_redo_floors(snapshot.catalog_replay_start_ts); assert_eq!(live_before_catalog_checkpoint.len(), 1); assert_eq!(live_before_catalog_checkpoint[0].floor, root_floor); assert!( recovered + .inner() + .core .catalog() .storage .checkpointed_silent_watermarks() @@ -3018,6 +3103,8 @@ mod tests { ); let session = recovered.new_session().unwrap(); let replayed_watermark = recovered + .inner() + .core .catalog() .storage .table_replay_silent_watermarks() @@ -3044,8 +3131,15 @@ mod tests { )) .await .unwrap(); - let snapshot = recovered.catalog().storage.checkpoint_snapshot(); + let snapshot = recovered + .inner() + .core + .catalog() + .storage + .checkpoint_snapshot(); let (live_after_catalog_checkpoint, _) = recovered + .inner() + .core .catalog() .snapshot_user_table_redo_floors(snapshot.catalog_replay_start_ts); assert_eq!(live_after_catalog_checkpoint.len(), 1); @@ -3092,12 +3186,20 @@ mod tests { .await .unwrap(); let catalog_replay_start_ts = engine + .inner() + .core .catalog() .storage .checkpoint_snapshot() .catalog_replay_start_ts; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut trx = session.begin_trx().unwrap(); let insert = trx_insert_row( &mut trx, @@ -3141,7 +3243,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); assert_eq!(session.total_row_pages(table.table_id()).await.unwrap(), 0); @@ -3222,7 +3330,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut trx = session.begin_trx().unwrap(); let insert = trx_insert_row( &mut trx, @@ -3283,7 +3397,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); assert!(session.total_row_pages(table.table_id()).await.unwrap() > 0); @@ -3360,7 +3480,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut same_row_ids = Vec::new(); let mut trx = session.begin_trx().unwrap(); for id in [1u32, 2, 3] { @@ -3407,7 +3533,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); assert_eq!(session.total_row_pages(table.table_id()).await.unwrap(), 0); @@ -3646,13 +3778,21 @@ mod tests { .await .unwrap(); let catalog_replay_start_ts = engine + .inner() + .core .catalog() .storage .checkpoint_snapshot() .catalog_replay_start_ts; assert!(catalog_replay_start_ts > MIN_SNAPSHOT_TS); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut trx = session.begin_trx().unwrap(); let insert = trx_insert_row( @@ -3704,7 +3844,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); assert!(session.total_row_pages(table.table_id()).await.unwrap() > 0); @@ -3771,7 +3917,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut trx = session.begin_trx().unwrap(); for i in 0..10u32 { let insert = trx_insert_row(&mut trx, &table, vec![Val::from(i)]).await; @@ -3838,7 +3990,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); assert_eq!( table.file().active_root_unchecked().deletion_cutoff_ts, checkpointed_cutoff @@ -3934,6 +4092,8 @@ mod tests { .await .unwrap(); let baseline_catalog_replay_start_ts = engine + .inner() + .core .catalog() .storage .checkpoint_snapshot() @@ -3941,11 +4101,15 @@ mod tests { assert!(baseline_catalog_replay_start_ts > MIN_SNAPSHOT_TS); let checkpointed_table = engine + .inner() + .core .catalog() .get_table(checkpointed_table_id) .await .unwrap(); let replay_only_table = engine + .inner() + .core .catalog() .get_table(replay_only_table_id) .await @@ -4010,6 +4174,8 @@ mod tests { .await .unwrap(); let final_catalog_replay_start_ts = engine + .inner() + .core .catalog() .storage .checkpoint_snapshot() @@ -4037,11 +4203,15 @@ mod tests { .unwrap(); let checkpointed_table = engine + .inner() + .core .catalog() .get_table(checkpointed_table_id) .await .unwrap(); let replay_only_table = engine + .inner() + .core .catalog() .get_table(replay_only_table_id) .await @@ -4129,7 +4299,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut trx = session.begin_trx().unwrap(); let insert = trx_insert_row( &mut trx, @@ -4192,7 +4368,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); let mut trx = session.begin_trx().unwrap(); let key = SelectKey::new(0, vec![Val::from(7u32)]); @@ -4254,7 +4436,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut trx = session.begin_trx().unwrap(); for i in 0..80u32 { let insert = trx_insert_row(&mut trx, &table, vec![Val::from(i)]).await; @@ -4346,7 +4534,13 @@ mod tests { .await .unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let session = engine.new_session().unwrap(); let active_root = table.file().active_root_unchecked(); { diff --git a/doradb-storage/src/recovery/resources.rs b/doradb-storage/src/recovery/resources.rs index 48153467..138f4f06 100644 --- a/doradb-storage/src/recovery/resources.rs +++ b/doradb-storage/src/recovery/resources.rs @@ -33,7 +33,7 @@ impl<'a> RecoveryResources<'a> { table_fs: QuiescentGuard, catalog: &'a Catalog, ) -> Self { - let pool_guards = pools.pool_guards(); + let pool_guards = pools.pool_guards().clone(); Self { pools, pool_guards, diff --git a/doradb-storage/src/session.rs b/doradb-storage/src/session.rs index cd5eef2c..080b54b3 100644 --- a/doradb-storage/src/session.rs +++ b/doradb-storage/src/session.rs @@ -1,7 +1,7 @@ use crate::buffer::page::VersionedPageID; use crate::buffer::{BufferPool, PoolGuards}; use crate::catalog::{ - CatalogCheckpointOutcome, CatalogCheckpointScope, CreateIndexPlan, DropIndexPlan, + Catalog, CatalogCheckpointOutcome, CatalogCheckpointScope, CreateIndexPlan, DropIndexPlan, DropTablePlan, IndexDdlGateScope, IndexNo, IndexSpec, PreparedCreateIndex, PreparedCreateTable, PreparedDropIndex, PreparedDropTable, TableSpec, ValidatedCreateTable, create_index_catalog_write_targets, create_table_catalog_write_targets, @@ -9,7 +9,7 @@ use crate::catalog::{ prepare_catalog_checkpoint_operation, reject_non_user_table_id, reject_user_table_primary_key_index, validated_index_ddl_target, }; -use crate::engine::{EngineInner, EngineRef, WeakEngineRef}; +use crate::engine::{EngineAdmission, EngineCore, EngineLifecycle}; use crate::error::{ CompletionErrorBridge, CompletionResult, DiscloseError, DiscloseResultExt, FatalError, LifecycleError, LifecycleResult, MultiDomainResultExt, OperationError, OperationResult, Result, @@ -47,6 +47,7 @@ use std::fmt::Display; use std::future::Future; use std::marker::PhantomData; use std::mem::replace; +use std::ops::Deref; use std::sync::atomic::{AtomicU64, Ordering}; use std::sync::{Arc, Weak}; @@ -123,7 +124,7 @@ impl PreparedDdlLocks { #[inline] fn new(operation: &SessionOperationPin) -> Self { Self { - lock_manager: operation.engine.lock_manager().clone(), + lock_manager: operation.runtime.lock_manager().clone(), locks: OwnerLockState::new(operation.operation_lock_owner()), } } @@ -282,14 +283,8 @@ impl PreparedDdlScope { /// Return the retained engine while caller preparation still owns the scope. #[inline] - pub(crate) fn engine(&self) -> &EngineRef { - &self.operation.engine - } - - /// Return cloned buffer-pool guards while caller preparation owns the scope. - #[inline] - pub(crate) fn pool_guards(&self) -> PoolGuards { - self.operation.pool_guards() + pub(crate) fn engine(&self) -> &SessionRuntime { + &self.operation.runtime } /// Synchronously consume caller preparation into accepted authority. @@ -320,14 +315,8 @@ pub(crate) struct AcceptedDdlScope { impl AcceptedDdlScope { /// Return the retained engine runtime. #[inline] - pub(crate) fn engine(&self) -> &EngineRef { - &self.operation.engine - } - - /// Return cloned buffer-pool guards for catalog/table lifecycle work. - #[inline] - pub(crate) fn pool_guards(&self) -> PoolGuards { - self.operation.state.pool_guards().clone() + pub(crate) fn engine(&self) -> &SessionRuntime { + &self.operation.runtime } /// Start one mandatory-owned nested private transaction. @@ -367,7 +356,7 @@ impl AcceptedDdlScope { self.operation.fail_retained(); let report = Report::new(FatalError::MandatoryTaskPanic) .attach("accepted table DDL finished without terminal-ready state"); - self.operation.engine.poisoner.poison(report); + self.operation.runtime.poisoner.poison(report); drop(self.locks.take()); } DdlFinishState::FailedRetained => { @@ -394,7 +383,7 @@ impl PreparedMaintenanceLocks { #[inline] fn new(operation: &SessionOperationPin) -> Self { Self { - lock_manager: operation.engine.lock_manager().clone(), + lock_manager: operation.runtime.lock_manager().clone(), locks: OwnerLockState::new(operation.operation_lock_owner()), } } @@ -465,8 +454,8 @@ impl PreparedMaintenanceScope { /// Return the retained engine while caller preparation owns the scope. #[inline] - pub(crate) fn engine(&self) -> &EngineRef { - &self.operation.engine + pub(crate) fn engine(&self) -> &SessionRuntime { + &self.operation.runtime } /// Resolve and retain the authoritative current-live table under locks. @@ -476,11 +465,11 @@ impl PreparedMaintenanceScope { ) -> OperationResult> { let table = self .operation - .engine + .runtime .catalog() .validate_user_table_live(table_id) .await?; - self.operation.state.cache_user_table(&table); + self.operation.runtime.state().cache_user_table(&table); Ok(table) } @@ -512,14 +501,14 @@ pub(crate) struct AcceptedMaintenanceScope { impl AcceptedMaintenanceScope { /// Return the retained engine runtime. #[inline] - pub(crate) fn engine(&self) -> &EngineRef { - &self.operation.engine + pub(crate) fn engine(&self) -> &SessionRuntime { + &self.operation.runtime } /// Return cloned buffer-pool guards for maintenance work. #[inline] - pub(crate) fn pool_guards(&self) -> PoolGuards { - self.operation.state.pool_guards().clone() + pub(crate) fn pool_guards(&self) -> &PoolGuards { + self.operation.runtime.pool_guards() } /// Start one mandatory-owned nested private transaction. @@ -551,7 +540,7 @@ impl AcceptedMaintenanceScope { self.operation.fail_retained(); let report = Report::new(FatalError::MandatoryTaskPanic) .attach("accepted maintenance finished without terminal-ready state"); - self.operation.engine.poisoner.poison(report); + self.operation.runtime.poisoner.poison(report); drop(self.locks.take()); } MaintenanceFinishState::FailedRetained => { @@ -570,13 +559,8 @@ impl AcceptedMaintenanceScope { impl SessionRuntimeAccess for AcceptedMaintenanceScope { #[inline] - fn engine(&self) -> &EngineRef { - &self.operation.engine - } - - #[inline] - fn state(&self) -> &Arc { - &self.operation.state + fn runtime(&self) -> &SessionRuntime { + &self.operation.runtime } } @@ -748,17 +732,17 @@ impl MaintenanceBoundary { #[inline] fn observed(self, session: &SessionObserverPin) -> TrxID { match self { - MaintenanceBoundary::GcHorizon => session.engine.trx_sys.published_gc_horizon(), - MaintenanceBoundary::PurgeCompletion => session.engine.trx_sys.global_visible_sts(), + MaintenanceBoundary::GcHorizon => session.runtime.trx_sys.published_gc_horizon(), + MaintenanceBoundary::PurgeCompletion => session.runtime.trx_sys.global_visible_sts(), } } #[inline] fn listener(self, session: &SessionObserverPin) -> event_listener::EventListener { match self { - MaintenanceBoundary::GcHorizon => session.engine.trx_sys.gc_horizon_listener(), + MaintenanceBoundary::GcHorizon => session.runtime.trx_sys.gc_horizon_listener(), MaintenanceBoundary::PurgeCompletion => { - session.engine.trx_sys.purge_completion_listener() + session.runtime.trx_sys.purge_completion_listener() } } } @@ -804,7 +788,7 @@ impl<'lock> ScopedTableRuntimeAccess<'lock> { owner: LockOwner, ) -> OperationResult<(Option>, Option>)> { session - .engine + .runtime .lock_manager() .acquire_table_locks(table_id, LockMode::IntentShared, owner) .await @@ -830,11 +814,201 @@ impl Drop for ScopedTableRuntimeAccess<'_> { } } +/// Limited per-session façade over the engine lifecycle admission gate. +pub(crate) struct SessionAdmission { + lifecycle: Arc, +} + +impl SessionAdmission { + /// Create one façade shared by a session state and its public handles. + #[inline] + pub(crate) fn new(lifecycle: Arc) -> Self { + Self { lifecycle } + } + + /// Acquire short-lived operation-start admission. + #[inline] + fn acquire(&self) -> LifecycleResult> { + self.lifecycle + .admit() + .attach_with(|| "phase=acquire_engine_lifecycle_admission") + } + + /// Returns whether owner-side shutdown has started. + #[inline] + pub(crate) fn shutdown_started(&self) -> bool { + self.lifecycle.shutdown_started() + } + + /// Register for owner-side shutdown start. + #[inline] + pub(crate) fn shutdown_listener(&self) -> EventListener { + self.lifecycle.shutdown_listener() + } +} + +/// Weak reachability to one exact registered session state. +#[derive(Clone)] +pub(crate) struct WeakSessionRef { + state: Weak, + admission: Arc, +} + +impl WeakSessionRef { + /// Create a weak session capability without retaining engine components. + #[inline] + fn new(state: &Arc) -> Self { + Self { + state: Arc::downgrade(state), + admission: Arc::clone(&state.admission), + } + } + + /// Acquire operation-start admission without cloning the admission façade. + #[inline] + pub(crate) fn acquire_admission(&self) -> LifecycleResult> { + let admission = self.admission.acquire()?; + Ok(AdmittedSessionRef { + state: &self.state, + _admission: admission, + }) + } + + /// Best-effort upgrade for terminal and cleanup ownership. + #[inline] + pub(crate) fn upgrade_for_terminal(&self) -> Option { + self.state.upgrade().map(SessionRuntime) + } +} + +/// Weak session reachability paired with its exact operation-start admission. +/// +/// The wrapper prevents normal weak upgrades from being separated from the +/// matching engine admission. It remains live until the caller registers a +/// stable operation or observer proof. +pub(crate) struct AdmittedSessionRef<'a> { + state: &'a Weak, + _admission: EngineAdmission<'a>, +} + +impl<'a> AdmittedSessionRef<'a> { + /// Consume this admitted weak reference and pin its exact session runtime. + #[inline] + pub(crate) fn upgrade(self) -> Option> { + let runtime = self.state.upgrade().map(SessionRuntime)?; + Some(AdmittedSessionRuntime { + runtime, + _admission: self._admission, + }) + } +} + +/// Strong session reachability retaining its operation-start admission. +/// +/// Callers release admission with [`Self::into_runtime`] only after registering +/// a stable operation or observer proof. +pub(crate) struct AdmittedSessionRuntime<'a> { + runtime: SessionRuntime, + _admission: EngineAdmission<'a>, +} + +impl AdmittedSessionRuntime<'_> { + /// Borrow the pinned runtime while admission remains active. + #[inline] + pub(crate) fn runtime(&self) -> &SessionRuntime { + &self.runtime + } + + /// Retain the runtime while releasing operation-start admission. + #[inline] + pub(crate) fn into_runtime(self) -> SessionRuntime { + self.runtime + } +} + +/// Strong operation-local reachability to one exact session state. +/// +/// This typed `Arc` wrapper pins the state reached by a public weak handle. +/// Engine capabilities are reached through the state without a separate core +/// clone. +#[derive(Clone)] +pub(crate) struct SessionRuntime(Arc); + +impl SessionRuntime { + /// Wrap one registered session state without another allocation. + #[inline] + pub(crate) fn new(state: Arc) -> Self { + Self(state) + } + + /// Return the exact pinned session state. + #[inline] + pub(crate) fn state(&self) -> &Arc { + &self.0 + } + + /// Return the immutable engine capability set reached through the state. + #[inline] + pub(crate) fn core(&self) -> &EngineCore { + &self.0.core + } + + /// Create weak reachability for a public transaction facade. + #[inline] + pub(crate) fn downgrade(&self) -> WeakSessionRef { + WeakSessionRef::new(&self.0) + } + + /// Borrow the canonical engine pool guard bundle. + #[inline] + pub(crate) fn pool_guards(&self) -> &PoolGuards { + self.core().pool_guards() + } + + /// Returns whether owner-side shutdown has closed operation admission. + #[inline] + pub(crate) fn shutdown_started(&self) -> bool { + self.0.admission.shutdown_started() + } + + /// Register for owner-side shutdown start through the session admission façade. + #[inline] + pub(crate) fn shutdown_listener(&self) -> EventListener { + self.0.admission.shutdown_listener() + } + + /// Clone catalog capability for an accepted ownership handoff. + #[inline] + pub(crate) fn catalog_guard(&self) -> QuiescentGuard { + self.core().catalog.clone() + } + + /// Remove this state only when the registry still owns this exact Arc. + #[inline] + fn remove_if_requested(&self, remove_from_registry: bool) { + if !remove_from_registry { + return; + } + if let Some(registry) = self.core().session_registry.upgrade() { + registry.remove_exact(self); + } + } +} + +impl Deref for SessionRuntime { + type Target = EngineCore; + + #[inline] + fn deref(&self) -> &Self::Target { + self.core() + } +} + /// Weak, non-cloneable public session capability bound to one engine instance. /// /// The engine owns the strong session state in its internal session registry. -/// Public session operations upgrade weak engine reachability internally, pin -/// the registry-owned state for one operation, and release registry/admission +/// Public session operations upgrade weak state reachability internally, pin +/// that exact state for one operation, and release admission /// guards before async work. A session may move between threads but cannot be /// shared between them. Lock-free observations use shared access; state /// mutation and every logical-lock transition require mutable access. Every @@ -843,7 +1017,7 @@ impl Drop for ScopedTableRuntimeAccess<'_> { /// is active. pub struct Session { id: SessionID, - engine: WeakEngineRef, + session: WeakSessionRef, /// Local explicit-close marker. /// /// `Cell` intentionally preserves `Send` while suppressing the `Sync` @@ -854,10 +1028,10 @@ pub struct Session { impl Session { /// Creates a weak public session handle. #[inline] - pub(crate) fn new(engine: WeakEngineRef, id: SessionID) -> Self { + pub(crate) fn new(session: WeakSessionRef, id: SessionID) -> Self { Session { id, - engine, + session, closed: Cell::new(false), } } @@ -878,16 +1052,27 @@ impl Session { return Err(Report::new(LifecycleError::SessionUnavailable) .attach(format!("session_id={}", self.id))); } - let engine = self - .engine - .upgrade() - .attach_with(|| format!("session_id={}, phase=upgrade_engine_runtime", self.id))?; - let admission = engine + let admitted = self + .session .acquire_admission() .attach_with(|| format!("session_id={}", self.id))?; - let state = engine.session_registry.pin_observer(self.id)?; - drop(admission); - Ok(SessionObserverPin { engine, state }) + let admitted = admitted.upgrade().ok_or_else(|| { + Report::new(LifecycleError::SessionUnavailable) + .attach(format!("session_id={}, reason=session_missing", self.id)) + })?; + admitted + .runtime() + .poisoner + .ensure_healthy() + .change_context(LifecycleError::RuntimeUnavailable) + .attach_with(|| format!("session_id={}, phase=check_engine_health", self.id))?; + admitted + .runtime() + .state() + .acquire_observer() + .attach_with(|| format!("session_id={}", self.id))?; + let runtime = admitted.into_runtime(); + Ok(SessionObserverPin { runtime }) } /// Reserves one stable entry for an effectful public session operation. @@ -897,21 +1082,34 @@ impl Session { return Err(Report::new(LifecycleError::SessionUnavailable) .attach(format!("session_id={}", self.id))); } - let engine = self.engine.upgrade().attach_with(|| { - format!( - "session_id={}, kind={}, phase=upgrade_engine_runtime", - self.id, - kind.label() - ) - })?; - let admission = engine + let admitted = self + .session .acquire_admission() .attach_with(|| format!("session_id={}, kind={}", self.id, kind.label()))?; - let (state, entry) = engine.session_registry.reserve_operation(self.id, kind)?; - drop(admission); + let admitted = admitted.upgrade().ok_or_else(|| { + Report::new(LifecycleError::SessionUnavailable) + .attach(format!("session_id={}, reason=session_missing", self.id)) + })?; + admitted + .runtime() + .poisoner + .ensure_healthy() + .change_context(LifecycleError::RuntimeUnavailable) + .attach_with(|| { + format!( + "session_id={}, kind={}, phase=check_engine_health", + self.id, + kind.label() + ) + })?; + let entry = admitted + .runtime() + .state() + .reserve_operation(kind) + .attach_with(|| format!("session_id={}, kind={}", self.id, kind.label()))?; + let runtime = admitted.into_runtime(); Ok(SessionOperationPin { - engine, - state, + runtime, entry, armed: true, }) @@ -929,16 +1127,21 @@ impl Session { return Err(Report::new(LifecycleError::SessionUnavailable) .attach(format!("session_id={}", self.id))); } - let engine = self - .engine - .upgrade() - .attach_with(|| format!("session_id={}, phase=upgrade_engine_runtime", self.id))?; - let admission = engine - .acquire_inspection_admission() + let admitted = self + .session + .acquire_admission() .attach_with(|| format!("session_id={}", self.id))?; - let state = engine.session_registry.pin_observer(self.id)?; - drop(admission); - Ok(SessionObserverPin { engine, state }) + let admitted = admitted.upgrade().ok_or_else(|| { + Report::new(LifecycleError::SessionUnavailable) + .attach(format!("session_id={}, reason=session_missing", self.id)) + })?; + admitted + .runtime() + .state() + .acquire_observer() + .attach_with(|| format!("session_id={}", self.id))?; + let runtime = admitted.into_runtime(); + Ok(SessionObserverPin { runtime }) } /// Return sorted ids for currently loaded user-table runtimes. @@ -953,7 +1156,7 @@ impl Session { .pin_inspection() .attach("operation=list_table_ids") .disclose()?; - Ok(session.engine.catalog().list_user_table_ids_now()) + Ok(session.runtime.catalog().list_user_table_ids_now()) } /// Begin a new transaction if the session is currently idle. @@ -964,21 +1167,32 @@ impl Session { .attach(format!("session_id={}", self.id)) .disclose()); } - let engine = self - .engine - .upgrade() - .attach_with(|| format!("session_id={}, phase=upgrade_engine_runtime", self.id)) - .disclose()?; - let admission = engine + let admitted = self + .session .acquire_admission() .attach_with(|| format!("session_id={}", self.id)) .disclose()?; - let trx = engine - .session_registry - .begin_public_transaction(self.id, &engine) + let admitted = admitted + .upgrade() + .ok_or_else(|| { + Report::new(LifecycleError::SessionUnavailable) + .attach(format!("session_id={}, reason=session_missing", self.id)) + }) + .disclose()?; + admitted + .runtime() + .poisoner + .ensure_healthy() + .change_context(LifecycleError::RuntimeUnavailable) + .attach_with(|| format!("session_id={}, phase=check_engine_health", self.id)) + .disclose()?; + let trx = admitted + .runtime() + .state() + .begin_public_trx(admitted.runtime()) .attach("operation=begin_transaction") .disclose()?; - drop(admission); + drop(admitted); Ok(trx) } @@ -988,27 +1202,44 @@ impl Session { if self.closed.get() { return Ok(()); } - let engine = self - .engine - .upgrade() - .attach_with(|| { - format!( - "operation=close_session, session_id={}, phase=upgrade_engine_runtime", - self.id - ) - }) - .disclose()?; - let admission = engine - .acquire_admission() - .attach_with(|| format!("operation=close_session, session_id={}", self.id)) - .disclose()?; - drop(admission); - engine - .session_registry - .close(self.id) - .await - .attach("operation=close_session") - .disclose()?; + let runtime = { + let admitted = self + .session + .acquire_admission() + .attach_with(|| format!("operation=close_session, session_id={}", self.id)) + .disclose()?; + let admitted = admitted + .upgrade() + .ok_or_else(|| { + Report::new(LifecycleError::SessionUnavailable) + .attach(format!("session_id={}, reason=session_missing", self.id)) + }) + .disclose()?; + admitted + .runtime() + .poisoner + .ensure_healthy() + .change_context(LifecycleError::RuntimeUnavailable) + .attach_with(|| { + format!( + "operation=close_session, session_id={}, phase=check_engine_health", + self.id + ) + }) + .disclose()?; + admitted.into_runtime() + }; + loop { + let (decision, remove_from_registry) = runtime.state().request_close(); + runtime.remove_if_requested(remove_from_registry); + match decision { + SessionCloseDecision::Closed => break, + SessionCloseDecision::Wait(listener) => listener.await, + SessionCloseDecision::Rejected(err) => { + return Err(err.attach("operation=close_session").disclose()); + } + } + } self.closed.set(true); Ok(()) } @@ -1025,7 +1256,7 @@ impl Session { .pin_operation(SessionOperationKind::Ddl) .attach("operation=create_table") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let prepared = operation .prepare_create_table(validated) .await @@ -1053,10 +1284,10 @@ impl Session { .pin_operation(SessionOperationKind::Ddl) .attach("operation=create_index") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let owner = operation.operation_lock_owner(); operation - .engine + .runtime .lock_manager() .reject_table_ddl_explicit_session_lock(table_id, owner) .attach("operation=create_index") @@ -1069,9 +1300,8 @@ impl Session { .await .attach_with(|| format!("prepare CREATE INDEX locks: table_id={table_id}")) .disclose()?; - let engine = scope.engine().clone(); - let guards = scope.pool_guards(); - let table = validated_index_ddl_target(&guards, &engine, table_id, "create_index") + let engine = scope.engine(); + let table = validated_index_ddl_target(engine, table_id, "create_index") .await .disclose()?; engine.poisoner.ensure_healthy().disclose()?; @@ -1097,10 +1327,10 @@ impl Session { .pin_operation(SessionOperationKind::Ddl) .attach("operation=drop_index") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let owner = operation.operation_lock_owner(); operation - .engine + .runtime .lock_manager() .reject_table_ddl_explicit_session_lock(table_id, owner) .attach("operation=drop_index") @@ -1110,9 +1340,8 @@ impl Session { .await .attach_with(|| format!("prepare DROP INDEX locks: table_id={table_id}")) .disclose()?; - let engine = scope.engine().clone(); - let guards = scope.pool_guards(); - let table = validated_index_ddl_target(&guards, &engine, table_id, "drop_index") + let engine = scope.engine(); + let table = validated_index_ddl_target(engine, table_id, "drop_index") .await .disclose()?; engine.poisoner.ensure_healthy().disclose()?; @@ -1138,7 +1367,7 @@ impl Session { .pin_operation(SessionOperationKind::Ddl) .attach("operation=drop_table") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let prepared = operation .prepare_drop_table(table_id) .await @@ -1165,13 +1394,13 @@ impl Session { .pin_operation(SessionOperationKind::Maintenance) .attach("operation=checkpoint_catalog") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let scope = PreparedMaintenanceScope::global(operation); - let engine = scope.engine().clone(); + let engine = scope.engine(); let catalog_scope = CatalogCheckpointScope::acquire(engine.catalog_guard()).await; let redo_scope = RedoRetentionScope::acquire(engine.trx_sys.clone()).await; - let prepared = prepare_catalog_checkpoint_operation(catalog_scope, redo_scope, scope); engine.poisoner.ensure_healthy().disclose()?; + let prepared = prepare_catalog_checkpoint_operation(catalog_scope, redo_scope, scope); let observer = mandatory_runtime .submit(prepared) .await @@ -1194,13 +1423,13 @@ impl Session { .pin_operation(SessionOperationKind::Maintenance) .attach("operation=checkpoint_catalog_and_truncate_redo_log") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let scope = PreparedMaintenanceScope::global(operation); - let engine = scope.engine().clone(); + let engine = scope.engine(); let catalog_scope = CatalogCheckpointScope::acquire(engine.catalog_guard()).await; let redo_scope = RedoRetentionScope::acquire(engine.trx_sys.clone()).await; - let prepared = prepare_catalog_redo_maintenance_operation(catalog_scope, redo_scope, scope); engine.poisoner.ensure_healthy().disclose()?; + let prepared = prepare_catalog_redo_maintenance_operation(catalog_scope, redo_scope, scope); let observer = mandatory_runtime .submit(prepared) .await @@ -1222,13 +1451,13 @@ impl Session { .pin_operation(SessionOperationKind::Maintenance) .attach("operation=truncate_redo_log") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let scope = PreparedMaintenanceScope::global(operation); - let engine = scope.engine().clone(); + let engine = scope.engine(); let catalog_scope = CatalogCheckpointScope::acquire(engine.catalog_guard()).await; let redo_scope = RedoRetentionScope::acquire(engine.trx_sys.clone()).await; - let prepared = prepare_redo_truncation_operation(catalog_scope, redo_scope, scope); engine.poisoner.ensure_healthy().disclose()?; + let prepared = prepare_redo_truncation_operation(catalog_scope, redo_scope, scope); let observer = mandatory_runtime .submit(prepared) .await @@ -1250,7 +1479,7 @@ impl Session { .pin_inspection() .attach("operation=query_transaction_system_stats") .disclose()?; - let engine = &session.engine; + let engine = &session.runtime; Ok(transaction_system_stats_snapshot( engine.trx_sys.trx_sys_stats(), )) @@ -1268,7 +1497,7 @@ impl Session { .pin_inspection() .attach("operation=query_storage_io_stats") .disclose()?; - let engine = &session.engine; + let engine = &session.runtime; Ok(storage_io_stats_snapshot( engine.table_fs.io_backend_stats(), engine.table_fs.storage_service_stats(), @@ -1287,27 +1516,27 @@ impl Session { .pin_inspection() .attach("operation=query_buffer_pool_stats") .disclose()?; - let engine = &session.engine; + let engine = &session.runtime; Ok(BufferPoolStats { meta: buffer_pool_runtime_stats_snapshot( - engine.meta_pool.capacity(), - engine.meta_pool.allocated(), - engine.meta_pool.stats(), + engine.pools.meta.capacity(), + engine.pools.meta.allocated(), + engine.pools.meta.stats(), ), mem: buffer_pool_runtime_stats_snapshot( - engine.mem_pool.capacity(), - engine.mem_pool.allocated(), - engine.mem_pool.stats(), + engine.pools.mem.capacity(), + engine.pools.mem.allocated(), + engine.pools.mem.stats(), ), index: buffer_pool_runtime_stats_snapshot( - engine.index_pool.capacity(), - engine.index_pool.allocated(), - engine.index_pool.stats(), + engine.pools.index.capacity(), + engine.pools.index.allocated(), + engine.pools.index.stats(), ), disk: buffer_pool_runtime_stats_snapshot( - engine.disk_pool.capacity(), - engine.disk_pool.allocated(), - engine.disk_pool.stats(), + engine.pools.disk.capacity(), + engine.pools.disk.allocated(), + engine.pools.disk.stats(), ), }) } @@ -1324,7 +1553,7 @@ impl Session { .pin_inspection() .attach("operation=query_mandatory_runtime_stats") .disclose()?; - Ok(session.engine.mandatory_runtime.stats()) + Ok(session.runtime.mandatory_runtime.stats()) } /// Freeze a row-page prefix or report the existing table-owned batch. @@ -1338,7 +1567,7 @@ impl Session { .pin_operation(SessionOperationKind::Maintenance) .attach("operation=freeze_table") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let scope = PreparedMaintenanceScope::table(operation, table_id) .await .attach_with(|| format!("operation=freeze_table, table_id={table_id}")) @@ -1369,7 +1598,7 @@ impl Session { .pin_operation(SessionOperationKind::Maintenance) .attach("operation=checkpoint_table") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let scope = PreparedMaintenanceScope::table(operation, table_id) .await .attach_with(|| format!("operation=checkpoint_table, table_id={table_id}")) @@ -1407,13 +1636,13 @@ impl Session { | CheckpointDelayReason::FrozenPageCutoff { table_id, .. } => table_id, }; loop { - let Some(table) = session.engine.catalog().current_live_user_table(table_id) else { + let Some(table) = session.runtime.catalog().current_live_user_table(table_id) else { return Ok(()); }; if table.check_foreground_live().is_err() { return Ok(()); } - session.state.cache_user_table(&table); + session.runtime.state().cache_user_table(&table); let observation = table .checkpoint_retry_observation(&session, reason) .await @@ -1487,7 +1716,7 @@ impl Session { let guards = session.pool_guards(); access .table() - .total_row_pages(&guards) + .total_row_pages(guards) .await .attach_with(|| format!("operation=count_table_row_pages, table_id={table_id}")) .disclose() @@ -1508,7 +1737,7 @@ impl Session { .pin_operation(SessionOperationKind::Maintenance) .attach("operation=cleanup_secondary_mem_indexes") .disclose()?; - let mandatory_runtime = operation.engine.mandatory_runtime.clone(); + let mandatory_runtime = operation.runtime.mandatory_runtime.clone(); let scope = PreparedMaintenanceScope::table(operation, table_id) .await .attach("operation=cleanup_secondary_mem_indexes") @@ -1567,21 +1796,24 @@ impl Drop for Session { if self.closed.get() { return; } - if let Some(engine) = self.engine.upgrade_for_cleanup() { - engine.session_registry.abandon(self.id); + if let Some(runtime) = self.session.upgrade_for_terminal() { + let remove_from_registry = runtime.state().abandon(); + runtime.remove_if_requested(remove_from_registry); } } } /// Shared runtime view implemented by observer and foreground authorities. pub(crate) trait SessionRuntimeAccess { - /// Returns the retained shared engine access handle. - fn engine(&self) -> &EngineRef; - /// Returns registry-owned session state. - fn state(&self) -> &Arc; - /// Returns a cloned pool-guard bundle. - fn pool_guards(&self) -> PoolGuards { - self.state().pool_guards().clone() + /// Returns the retained exact session runtime. + fn runtime(&self) -> &SessionRuntime; + /// Returns immutable shared engine capabilities. + fn engine(&self) -> &EngineCore { + self.runtime().core() + } + /// Borrows the canonical pool-guard bundle. + fn pool_guards(&self) -> &PoolGuards { + self.runtime().pool_guards() } } @@ -1590,37 +1822,29 @@ pub(crate) trait SessionRuntimeAccess { /// The creating `Session` method establishes whether normal healthy-runtime or /// lifecycle-only inspection admission applies. pub(crate) struct SessionObserverPin { - /// Engine handle retained for the duration of this observation. - pub(crate) engine: EngineRef, - /// Strong reference to registry-owned session state. - pub(crate) state: Arc, + /// Exact state and engine capabilities retained for this observation. + pub(crate) runtime: SessionRuntime, } impl Drop for SessionObserverPin { #[inline] fn drop(&mut self) { - self.engine.session_registry.finish_observer(&self.state); + let remove_from_registry = self.runtime.state().release_observer(); + self.runtime.remove_if_requested(remove_from_registry); } } impl SessionRuntimeAccess for SessionObserverPin { #[inline] - fn engine(&self) -> &EngineRef { - &self.engine - } - - #[inline] - fn state(&self) -> &Arc { - &self.state + fn runtime(&self) -> &SessionRuntime { + &self.runtime } } /// Non-cloneable foreground authority for one stable session operation. pub(crate) struct SessionOperationPin { - /// Engine access retained while the stable operation blocks shutdown. - pub(crate) engine: EngineRef, - /// Registry-owned session state containing the active slot. - pub(crate) state: Arc, + /// Exact state and engine capabilities retained while the operation blocks shutdown. + pub(crate) runtime: SessionRuntime, /// Stable entry shared with transaction, cleanup, and terminal owners. entry: Arc, /// Whether drop must publish the foreground release edge. @@ -1663,8 +1887,8 @@ impl SessionOperationPin { /// Returns a cloned guard bundle for this foreground operation. #[inline] - pub(crate) fn pool_guards(&self) -> PoolGuards { - self.state.pool_guards().clone() + pub(crate) fn pool_guards(&self) -> &PoolGuards { + self.runtime.pool_guards() } /// Consume voluntary authority at the exact mandatory ownership handoff. @@ -1674,11 +1898,10 @@ impl SessionOperationPin { /// operation can replace that active identity before terminal publication. #[inline] pub(crate) fn into_mandatory(mut self) -> MandatoryOperationGuard { - self.state.accept_mandatory(&self.entry); + self.runtime.state().accept_mandatory(&self.entry); self.armed = false; MandatoryOperationGuard { - engine: self.engine.clone(), - state: Arc::clone(&self.state), + runtime: self.runtime.clone(), entry: Arc::clone(&self.entry), armed: true, } @@ -1689,7 +1912,7 @@ impl SessionOperationPin { self, validated: ValidatedCreateTable, ) -> OperationResult { - let table_id = self.engine.catalog().next_table_id(); + let table_id = self.runtime.catalog().next_table_id(); let plan = validated.into_plan(table_id); let scope = PreparedDdlScope::create(self, table_id, create_table_catalog_write_targets()) .await @@ -1700,7 +1923,7 @@ impl SessionOperationPin { /// Prepare DROP TABLE while consuming this foreground operation. async fn prepare_drop_table(self, table_id: TableID) -> OperationResult { let owner = self.operation_lock_owner(); - self.engine + self.runtime .lock_manager() .reject_table_ddl_explicit_session_lock(table_id, owner) .attach("prepare DROP TABLE explicit-session-lock check")?; @@ -1730,11 +1953,11 @@ impl SessionOperationPin { table_id: TableID, ) -> OperationResult> { let table = self - .engine + .runtime .catalog() .validate_user_table_live(table_id) .await?; - self.state.cache_user_table(&table); + self.runtime.state().cache_user_table(&table); Ok(table) } @@ -1746,7 +1969,7 @@ impl SessionOperationPin { mode: LockMode, ) -> OperationResult<()> { let session_id = self.id(); - let engine = &self.engine; + let engine = &self.runtime; let lock_manager = engine.lock_manager(); let owner = LockOwner::session_explicit(session_id); let (mut metadata_guard, mut data_guard) = lock_manager @@ -1766,7 +1989,7 @@ impl SessionOperationPin { #[inline] pub(crate) fn unlock_table(&self, table_id: TableID) -> OperationResult<()> { let owner = LockOwner::session_explicit(self.id()); - let lock_manager = self.engine.lock_manager(); + let lock_manager = self.runtime.lock_manager(); lock_manager.release(LockResource::TableData(table_id), owner); lock_manager.release(LockResource::TableMetadata(table_id), owner); Ok(()) @@ -1775,13 +1998,8 @@ impl SessionOperationPin { impl SessionRuntimeAccess for SessionOperationPin { #[inline] - fn engine(&self) -> &EngineRef { - &self.engine - } - - #[inline] - fn state(&self) -> &Arc { - &self.state + fn runtime(&self) -> &SessionRuntime { + &self.runtime } } @@ -1790,9 +2008,16 @@ impl Drop for SessionOperationPin { fn drop(&mut self) { if self.armed { self.armed = false; - self.engine - .session_registry - .finish_foreground(&self.engine, self.key()); + let (remove_from_registry, cleanup) = + self.runtime.state().finish_foreground(self.key()); + self.runtime.remove_if_requested(remove_from_registry); + if let Some(trx_id) = cleanup { + self.runtime.trx_sys.request_abandoned_trx_cleanup( + self.runtime.clone(), + self.key(), + trx_id, + ); + } } } } @@ -1806,8 +2031,7 @@ impl Drop for SessionOperationPin { /// Nested private-transaction state can therefore move directly through /// `entry` without locking the outer lifecycle. pub(crate) struct MandatoryOperationGuard { - engine: EngineRef, - state: Arc, + runtime: SessionRuntime, /// Intentionally redundant with the `Arc` retained by `Active(entry)`. /// /// This direct reference is the guard's exact operation authority. It @@ -1830,7 +2054,7 @@ impl MandatoryOperationGuard { /// installation needs only the entry mutex rather than the lifecycle lock. #[inline] pub(crate) fn begin_private_trx(&self) -> LifecycleResult { - begin_private_transaction(&self.engine, &self.entry) + begin_private_transaction(&self.runtime, &self.entry) } /// Verify that accepted execution settled every nested transaction. @@ -1847,10 +2071,8 @@ impl MandatoryOperationGuard { if !self.armed { return; } - let remove_from_registry = self.state.finish_mandatory(&self.entry); - self.engine - .session_registry - .remove_if_requested(self.key().session_id(), remove_from_registry); + let remove_from_registry = self.runtime.state().finish_mandatory(&self.entry); + self.runtime.remove_if_requested(remove_from_registry); self.armed = false; } @@ -1861,19 +2083,14 @@ impl MandatoryOperationGuard { return; } self.armed = false; - self.state.fail_mandatory_retained(&self.entry); + self.runtime.state().fail_mandatory_retained(&self.entry); } } impl SessionRuntimeAccess for MandatoryOperationGuard { #[inline] - fn engine(&self) -> &EngineRef { - &self.engine - } - - #[inline] - fn state(&self) -> &Arc { - &self.state + fn runtime(&self) -> &SessionRuntime { + &self.runtime } } @@ -1884,12 +2101,12 @@ impl Drop for MandatoryOperationGuard { return; } self.armed = false; - self.state.fail_mandatory_retained(&self.entry); + self.runtime.state().fail_mandatory_retained(&self.entry); let report = Report::new(FatalError::MandatoryTaskPanic).attach(format!( "mandatory operation authority dropped unexpectedly: operation_key={}", self.key() )); - self.engine.poisoner.poison(report); + self.runtime.poisoner.poison(report); } } @@ -1904,6 +2121,8 @@ pub(crate) enum SessionShutdownBlocker { /// The tuple locates the stable outer operation entry first and /// identifies its currently attached transaction second. cleanup: Option<(SessionOperationKey, TrxID)>, + /// Runtime captured from the registered state during the shutdown scan. + runtime: Option, }, /// One or more standalone read-only observers remain active. Observer { @@ -1942,14 +2161,45 @@ impl SessionShutdownBlocker { /// Returns an exact claimable cleanup hint, when one exists. #[inline] - pub(crate) const fn cleanup(&self) -> Option<(SessionOperationKey, TrxID)> { + pub(crate) fn into_cleanup(self) -> Option { match self { - Self::Operation { cleanup, .. } => *cleanup, + Self::Operation { + cleanup: Some((operation_key, trx_id)), + runtime: Some(runtime), + .. + } => Some(SessionCleanupRequest { + runtime, + operation_key, + trx_id, + }), Self::Observer { .. } => None, + Self::Operation { .. } => None, + } + } + + #[inline] + fn capture_runtime(&mut self, runtime: SessionRuntime) { + if let Self::Operation { + cleanup: Some(_), + runtime: captured, + .. + } = self + { + *captured = Some(runtime); } } } +/// Exact shutdown-discovered cleanup authority captured during registry scan. +pub(crate) struct SessionCleanupRequest { + /// Exact registered session state and engine capabilities. + pub(crate) runtime: SessionRuntime, + /// Stable operation identity. + pub(crate) operation_key: SessionOperationKey, + /// Exact transaction identity. + pub(crate) trx_id: TrxID, +} + /// One session-local blocker observed by blocking shutdown. pub(crate) struct SessionShutdownWait { /// Listener installed before the selected blocker was re-read. @@ -1983,69 +2233,39 @@ impl SessionRegistry { #[inline] pub(crate) fn create_session( &self, - engine: &Arc, - engine_ref: EngineRef, + core: Arc, + admission: Arc, id: SessionID, ) -> Session { - let state = Arc::new(SessionState::new(engine_ref, id)); + let state = Arc::new(SessionState::new(core, admission, id)); + let session = Session::new(WeakSessionRef::new(&state), id); self.insert(state); - Session::new(WeakEngineRef::new(engine), id) - } - - /// Pins one open session for a drop-safe observer operation. - #[inline] - pub(crate) fn pin_observer(&self, id: SessionID) -> LifecycleResult> { - let state = self.session_or_unavailable(id)?; - state - .acquire_observer() - .attach_with(|| format!("session_id={id}"))?; - Ok(state) - } - - /// Releases one exact observer and removes its closed session if it drains. - #[inline] - pub(crate) fn finish_observer(&self, state: &Arc) { - let remove_from_registry = state.release_observer(); - if remove_from_registry { - self.entries.remove_if(&state.id(), |_id, registered| { - Arc::ptr_eq(registered, state) - }); - } - } - - /// Reserves one stable non-transaction operation entry. - #[inline] - pub(crate) fn reserve_operation( - &self, - id: SessionID, - kind: SessionOperationKind, - ) -> LifecycleResult<(Arc, Arc)> { - let state = self.session_or_unavailable(id)?; - let entry = state - .reserve_operation(kind) - .attach_with(|| format!("session_id={id}, kind={}", kind.label()))?; - Ok((state, entry)) + session } - /// Reserves and starts one public transaction under the lifecycle mutex. + /// Resolve one exact operation through the registry for legacy test setup. + #[cfg(test)] #[inline] - pub(crate) fn begin_public_transaction( + pub(crate) fn try_resolve_operation( &self, - id: SessionID, - engine: &EngineRef, - ) -> LifecycleResult { - let state = self.session_or_unavailable(id)?; - state.begin_public_trx(engine) + key: SessionOperationKey, + ) -> Option<(Arc, Arc)> { + let session = self.session_state(key.session_id())?; + let entry = session.resolve_operation(key)?; + Some((entry, session)) } - /// Explicitly closes this session, waiting only for authoritative owners. + /// Close one registry-owned session directly for lifecycle tests. + #[cfg(test)] pub(crate) async fn close(&self, id: SessionID) -> LifecycleResult<()> { loop { let Some(state) = self.session_state(id) else { return Ok(()); }; let (decision, remove_from_registry) = state.request_close(); - self.remove_if_requested(id, remove_from_registry); + if remove_from_registry { + self.remove_exact(&SessionRuntime::new(Arc::clone(&state))); + } match decision { SessionCloseDecision::Closed => return Ok(()), SessionCloseDecision::Wait(listener) => listener.await, @@ -2054,90 +2274,6 @@ impl SessionRegistry { } } - /// Best-effort nonblocking abandonment from public session `Drop`. - #[inline] - pub(crate) fn abandon(&self, id: SessionID) { - let Some(state) = self.session_state(id) else { - return; - }; - let remove_from_registry = state.abandon(); - self.remove_if_requested(id, remove_from_registry); - } - - /// Finalizes or transfers one dropped foreground operation authority. - #[inline] - pub(crate) fn finish_foreground(&self, engine: &EngineRef, key: SessionOperationKey) { - let Some(state) = self.session_state(key.session_id()) else { - return; - }; - let (remove_from_registry, cleanup) = state.finish_foreground(key); - self.remove_if_requested(key.session_id(), remove_from_registry); - if let Some(trx_id) = cleanup { - engine - .trx_sys - .request_abandoned_trx_cleanup(engine.clone(), key, trx_id); - } - } - - /// Apply session cleanup after a transaction commits. - #[inline] - fn finish_trx_commit(&self, key: SessionOperationKey, trx_id: TrxID, cts: TrxID) { - let Some(state) = self.session_state(key.session_id()) else { - return; - }; - let remove_from_registry = state.finish_trx_commit(key, trx_id, cts); - self.remove_if_requested(key.session_id(), remove_from_registry); - } - - /// Apply session cleanup after a transaction rolls back. - #[inline] - fn finish_trx_rollback(&self, key: SessionOperationKey, trx_id: TrxID) { - let Some(state) = self.session_state(key.session_id()) else { - return; - }; - let remove_from_registry = state.finish_trx_rollback(key, trx_id); - self.remove_if_requested(key.session_id(), remove_from_registry); - } - - /// Resolves one exact operation key before its entry transition validates - /// the transaction id under the entry mutex. - #[inline] - pub(crate) fn resolve_operation( - &self, - key: SessionOperationKey, - ) -> LifecycleResult<(Arc, Arc)> { - let session = self.session_state(key.session_id()).ok_or_else(|| { - Report::new(LifecycleError::TransactionDiscarded) - .attach(format!("operation_key={key}, reason=session_missing")) - })?; - let entry = session.resolve_operation(key).ok_or_else(|| { - Report::new(LifecycleError::TransactionDiscarded).attach(format!( - "operation_key={key}, reason=operation_entry_missing" - )) - })?; - Ok((entry, session)) - } - - /// Resolves an active operation without turning staleness into an error. - #[inline] - pub(crate) fn try_resolve_operation( - &self, - key: SessionOperationKey, - ) -> Option<(Arc, Arc)> { - let session = self.session_state(key.session_id())?; - let entry = session.resolve_operation(key)?; - Some((entry, session)) - } - - /// Mark a public transaction handle abandoned if it still names the active entry. - #[inline] - pub(crate) fn abandon_trx_handle(&self, key: SessionOperationKey, trx_id: TrxID) -> bool { - let Some(session) = self.session_state(key.session_id()) else { - return false; - }; - session.abandon_trx_handle(key, trx_id) - } - /// Returns the first active session operation without installing a listener. /// /// The DashMap iterator retains a shard read guard while each session takes @@ -2167,32 +2303,29 @@ impl SessionRegistry { let sessions = self .entries .iter() - .map(|entry| (*entry.key(), Arc::clone(entry.value()))) + .map(|entry| Arc::clone(entry.value())) .collect::>(); - for (id, state) in sessions { + for state in sessions { let remove_from_registry = state.shutdown_removal(); - self.remove_if_requested(id, remove_from_registry); + if remove_from_registry { + self.remove_exact(&SessionRuntime::new(state)); + } } } + #[cfg(test)] #[inline] fn session_state(&self, id: SessionID) -> Option> { self.entries.get(&id).map(|entry| Arc::clone(entry.value())) } + /// Remove only the pointer-identical registered state. #[inline] - fn session_or_unavailable(&self, id: SessionID) -> LifecycleResult> { - self.session_state(id).ok_or_else(|| { - Report::new(LifecycleError::SessionUnavailable) - .attach(format!("session_id={id}, reason=session_missing")) - }) - } - - #[inline] - fn remove_if_requested(&self, id: SessionID, remove_from_registry: bool) { - if remove_from_registry { - drop(self.entries.remove(&id)); - } + pub(crate) fn remove_exact(&self, runtime: &SessionRuntime) { + let state = runtime.state(); + self.entries.remove_if(&state.id(), |_id, registered| { + Arc::ptr_eq(registered, state) + }); } } @@ -2215,23 +2348,25 @@ impl SessionTableCacheEntry { /// Shared mutable state referenced by transactions started from one [`Session`]. pub(crate) struct SessionState { id: SessionID, - pool_guards: PoolGuards, - lock_manager: QuiescentGuard, + core: Arc, + admission: Arc, lifecycle: Mutex, last_cts: AtomicU64, table_cache: Mutex>, } impl SessionState { - /// Create a new session state and populate its default pool guards. + /// Create a new session state retaining one engine core and admission façade. #[inline] - pub(crate) fn new(engine_ref: EngineRef, id: SessionID) -> Self { - let pool_guards = engine_ref.pools().pool_guards(); - let lock_manager = engine_ref.lock_manager().clone(); + pub(crate) fn new( + core: Arc, + admission: Arc, + id: SessionID, + ) -> Self { SessionState { id, - pool_guards, - lock_manager, + core, + admission, lifecycle: Mutex::new(SessionLifecycle { disposition: SessionDisposition::Open, slot: SessionOperationSlot::Idle, @@ -2251,12 +2386,6 @@ impl SessionState { self.id } - /// Returns the guard bundle owned by this session state. - #[inline] - pub fn pool_guards(&self) -> &PoolGuards { - &self.pool_guards - } - #[inline] fn acquire_observer(&self) -> LifecycleResult<()> { let mut lifecycle = self.lifecycle.lock(); @@ -2364,7 +2493,10 @@ impl SessionState { } #[inline] - fn begin_public_trx(self: &Arc, engine: &EngineRef) -> LifecycleResult { + fn begin_public_trx( + self: &Arc, + runtime: &SessionRuntime, + ) -> LifecycleResult { let mut lifecycle = self.lifecycle.lock(); lifecycle .admit_idle() @@ -2376,7 +2508,9 @@ impl SessionState { self.id ) }); - let (trx, entry) = engine.trx_sys.begin_public_trx(engine, key, inner); + let (trx, entry) = runtime + .trx_sys + .begin_public_trx(runtime.downgrade(), key, inner); lifecycle.advance_operation_id(); lifecycle.slot = SessionOperationSlot::Active(entry); Ok(trx) @@ -2592,14 +2726,19 @@ impl SessionState { } } + /// Resolve an exact operation key directly on this pinned session state. #[inline] - fn resolve_operation(&self, key: SessionOperationKey) -> Option> { + pub(crate) fn resolve_operation( + &self, + key: SessionOperationKey, + ) -> Option> { let lifecycle = self.lifecycle.lock(); lifecycle.active_entry(key).cloned() } + /// Abandon the exact public transaction handle when its identity matches. #[inline] - fn abandon_trx_handle(&self, key: SessionOperationKey, trx_id: TrxID) -> bool { + pub(crate) fn abandon_trx_handle(&self, key: SessionOperationKey, trx_id: TrxID) -> bool { let lifecycle = self.lifecycle.lock(); let abandoned = lifecycle .active_entry(key) @@ -2628,19 +2767,22 @@ impl SessionState { } #[inline] - fn shutdown_blocker(&self) -> Option { + fn shutdown_blocker(self: &Arc) -> Option { let lifecycle = self.lifecycle.lock(); - lifecycle.shutdown_blocker() + let mut blocker = lifecycle.shutdown_blocker()?; + blocker.capture_runtime(SessionRuntime::new(Arc::clone(self))); + Some(blocker) } #[inline] - fn shutdown_wait(&self) -> Option { + fn shutdown_wait(self: &Arc) -> Option { let mut lifecycle = self.lifecycle.lock(); lifecycle.shutdown_blocker()?; let listener = lifecycle.change_listener(); - let blocker = lifecycle + let mut blocker = lifecycle .shutdown_blocker() .expect("session blocker cannot change while lifecycle lock is held"); + blocker.capture_runtime(SessionRuntime::new(Arc::clone(self))); Some(SessionShutdownWait { listener, blocker }) } @@ -2707,7 +2849,8 @@ impl SessionState { #[inline] fn release_session_locks(&self) { - self.lock_manager + self.core + .lock_manager() .release_owner(LockOwner::session_explicit(self.id)); } } @@ -2793,6 +2936,7 @@ impl SessionLifecycle { cleanup: entry .cleanup_candidate() .map(|trx_id| (entry.key(), trx_id)), + runtime: None, }); } (self.observer_count != 0).then_some(SessionShutdownBlocker::Observer { @@ -2895,51 +3039,43 @@ enum SessionCloseDecision { /// Private transaction runtime attachment retained by checked-out transaction work. /// -/// This handle owns engine access and session-state reachability for one +/// This handle owns exact session runtime reachability for one /// operation, terminal path, prepared commit handoff, or cleanup path. The /// stable session operation remains the shutdown proof, and the public /// transaction facade never stores this attachment. pub(crate) struct TrxAttachment { - /// Strong engine access kept until the transaction reaches a terminal path. - engine: EngineRef, - /// Exact registry lookup key for terminal session-operation cleanup. + /// Exact state and engine capabilities retained by this claim. + runtime: SessionRuntime, + /// Exact session-local operation key for terminal cleanup. operation_key: SessionOperationKey, /// Active transaction id used to avoid finishing a replaced session state. trx_id: TrxID, - /// Claim-local reachability for session-local caches. - session: Arc, - /// Cached guards needed by transaction work even if session state is gone. - pool_guards: PoolGuards, } impl TrxAttachment { /// Create a transaction runtime attachment without public handle ownership. #[inline] pub(crate) fn new( - engine: EngineRef, - session: Arc, + runtime: SessionRuntime, operation_key: SessionOperationKey, trx_id: TrxID, ) -> Self { - let pool_guards = session.pool_guards().clone(); assert!( - session.id() == operation_key.session_id(), + runtime.state().id() == operation_key.session_id(), "transaction attachment session/key mismatch: session_id={}, operation_key={operation_key}", - session.id() + runtime.state().id() ); Self { - engine, + runtime, operation_key, trx_id, - session, - pool_guards, } } - /// Returns the crate-private engine runtime handle. + /// Returns immutable engine capabilities. #[inline] - pub(crate) fn engine(&self) -> &EngineRef { - &self.engine + pub(crate) fn engine(&self) -> &EngineCore { + self.runtime.core() } /// Returns the authoritative session identity for transaction lock ownership. @@ -2955,28 +3091,30 @@ impl TrxAttachment { self.trx_id } - /// Returns the cloned session pool guards retained by this transaction. + /// Borrows the canonical engine pool guards. #[inline] pub(crate) fn pool_guards(&self) -> &PoolGuards { - &self.pool_guards + self.runtime.pool_guards() } /// Store a weak session-local table cache entry after successful resolution. #[inline] pub(crate) fn cache_user_table(&self, table: &Arc
) { - self.session.cache_user_table(table); + self.runtime.state().cache_user_table(table); } /// Remove and return the cached insert page for a table, if session state remains. #[inline] pub(crate) fn load_active_insert_page(&self, table_id: TableID) -> Option { - self.session.load_active_insert_page(table_id) + self.runtime.state().load_active_insert_page(table_id) } /// Cache the active insert page if session state remains. #[inline] pub(crate) fn save_active_insert_page(&self, table_id: TableID, page_id: VersionedPageID) { - self.session.save_active_insert_page(table_id, page_id); + self.runtime + .state() + .save_active_insert_page(table_id, page_id); } /// Mark the owning session committed. @@ -2988,23 +3126,27 @@ impl TrxAttachment { inner: Box, ) { released.assert_validated_for(self.trx_id); - self.session + self.runtime + .state() .finish_trx_inner(self.operation_key, self.trx_id, inner); #[cfg(test)] tests::run_terminal_attachment_test_hook( self.trx_id, tests::TerminalAttachmentOutcome::Commit, ); - self.engine - .session_registry - .finish_trx_commit(self.operation_key, self.trx_id, cts); + let remove_from_registry = + self.runtime + .state() + .finish_trx_commit(self.operation_key, self.trx_id, cts); + self.runtime.remove_if_requested(remove_from_registry); } /// Mark the owning session rolled back. #[inline] pub(crate) fn rollback(&self, released: ReleasedTransactionLocks, inner: Box) { released.assert_validated_for(self.trx_id); - self.session + self.runtime + .state() .finish_trx_inner(self.operation_key, self.trx_id, inner); self.finish_rollback(); } @@ -3023,16 +3165,18 @@ impl TrxAttachment { self.trx_id, tests::TerminalAttachmentOutcome::Rollback, ); - self.engine - .session_registry + let remove_from_registry = self + .runtime + .state() .finish_trx_rollback(self.operation_key, self.trx_id); + self.runtime.remove_if_requested(remove_from_registry); } /// Queue rollback cleanup for an abandoned transaction. #[inline] pub(crate) fn request_abandoned_cleanup(&self) { - self.engine.trx_sys.request_abandoned_trx_cleanup( - self.engine.clone(), + self.runtime.trx_sys.request_abandoned_trx_cleanup( + self.runtime.clone(), self.operation_key, self.trx_id, ); @@ -3041,14 +3185,16 @@ impl TrxAttachment { /// Notifies close or shutdown only when this exact operation was armed. #[inline] pub(crate) fn notify_operation_transition(&self) { - self.session.notify_operation_transition(self.operation_key); + self.runtime + .state() + .notify_operation_transition(self.operation_key); } } /// Starts one private transaction under an existing DDL or maintenance owner. #[inline] fn begin_private_transaction( - engine: &EngineRef, + runtime: &SessionRuntime, entry: &Arc, ) -> LifecycleResult { let kind = entry.kind(); @@ -3062,7 +3208,9 @@ fn begin_private_transaction( kind.label() ); let inner = Box::new(TrxInner::private()); - Ok(engine.trx_sys.begin_private_trx(engine, entry, inner)) + Ok(runtime + .trx_sys + .begin_private_trx(runtime.downgrade(), entry, inner)) } async fn wait_for_maintenance_boundary( @@ -3070,10 +3218,10 @@ async fn wait_for_maintenance_boundary( ts: TrxID, boundary: MaintenanceBoundary, ) -> Result { - let trx_sys = &session.engine.trx_sys; + let trx_sys = &session.runtime.trx_sys; loop { - session.engine.poisoner.ensure_healthy().disclose()?; - if session.engine.shutdown_started() { + session.runtime.poisoner.ensure_healthy().disclose()?; + if session.runtime.state().admission.shutdown_started() { return Err(Report::new(LifecycleError::Shutdown) .attach(format!( "maintenance progress wait observed engine shutdown: boundary={}, target_ts={ts}", @@ -3088,11 +3236,11 @@ async fn wait_for_maintenance_boundary( trx_sys.request_purge_observation(); let progress_listener = boundary.listener(session); - let poison_listener = session.engine.poisoner.listener(); - let shutdown_listener = session.engine.shutdown_listener(); + let poison_listener = session.runtime.poisoner.listener(); + let shutdown_listener = session.runtime.state().admission.shutdown_listener(); - session.engine.poisoner.ensure_healthy().disclose()?; - if session.engine.shutdown_started() { + session.runtime.poisoner.ensure_healthy().disclose()?; + if session.runtime.state().admission.shutdown_started() { return Err(Report::new(LifecycleError::Shutdown) .attach(format!( "maintenance progress wait observed engine shutdown: boundary={}, target_ts={ts}", @@ -3148,9 +3296,10 @@ pub(crate) mod tests { use std::path::{Path, PathBuf}; use std::pin::Pin; use std::sync::atomic::AtomicBool; - use std::sync::{Arc, Barrier, OnceLock}; + use std::sync::{Arc, Barrier, OnceLock, mpsc}; use std::task::{Context, Poll}; use std::thread; + use std::time::Duration; use tempfile::TempDir; const TRUNCATE_TEST_LOG_BLOCK_SIZE: usize = 4096; @@ -3255,7 +3404,12 @@ pub(crate) mod tests { panic!("test transaction requires active operation slot") } }; - registry.finish_trx_commit(key, trx_id, cts); + let remove_from_registry = state.finish_trx_commit(key, trx_id, cts); + if remove_from_registry { + registry.entries.remove_if(&session_id, |_id, registered| { + Arc::ptr_eq(registered, &state) + }); + } } type TotalRowPagesAfterAccessHook = @@ -3312,10 +3466,10 @@ pub(crate) mod tests { .pin_observer() .attach("operation=wait_for_purge_handoff") .disclose()?; - let trx_sys = &session.engine.trx_sys; + let trx_sys = &session.runtime.trx_sys; loop { - session.engine.poisoner.ensure_healthy().disclose()?; - if session.engine.shutdown_started() { + session.runtime.poisoner.ensure_healthy().disclose()?; + if session.runtime.shutdown_started() { return Err(Report::new(LifecycleError::Shutdown) .attach("completed-purge wait observed engine shutdown before ordered handoff") .disclose()); @@ -3324,10 +3478,10 @@ pub(crate) mod tests { return Ok(()); } let handoff_listener = trx_sys.purge_handoff_listener(); - let poison_listener = session.engine.poisoner.listener(); - let shutdown_listener = session.engine.shutdown_listener(); - session.engine.poisoner.ensure_healthy().disclose()?; - if session.engine.shutdown_started() { + let poison_listener = session.runtime.poisoner.listener(); + let shutdown_listener = session.runtime.shutdown_listener(); + session.runtime.poisoner.ensure_healthy().disclose()?; + if session.runtime.shutdown_started() { return Err(Report::new(LifecycleError::Shutdown) .attach("completed-purge wait observed engine shutdown before ordered handoff") .disclose()); @@ -3585,6 +3739,21 @@ pub(crate) mod tests { drop(registry.entries.remove(&session_id)); } + #[inline] + fn new_session_state_for_test(engine: &Engine, id: SessionID) -> SessionState { + let seed = engine.new_session().unwrap(); + let state = seed + .session + .state + .upgrade() + .expect("new test session must remain registered"); + let synthetic = + SessionState::new(Arc::clone(&state.core), Arc::clone(&state.admission), id); + drop(state); + drop(seed); + synthetic + } + #[inline] fn active_operation_entry_for_test( registry: &SessionRegistry, @@ -3605,14 +3774,14 @@ pub(crate) mod tests { /// Create one registry-owned transaction with test-controlled ids. #[inline] pub(crate) fn create_test_transaction( - registry: &SessionRegistry, - engine: EngineRef, + engine: &Engine, session_id: SessionID, trx_id: TrxID, sts: TrxID, gc_no: usize, ) -> (Transaction, Arc) { - let state = Arc::new(SessionState::new(engine.clone(), session_id)); + let registry = &engine.inner().session_registry; + let state = Arc::new(new_session_state_for_test(engine, session_id)); let key = SessionOperationKey::new(session_id, OperationID::new(1)); let mut inner = state .lifecycle @@ -3627,9 +3796,10 @@ pub(crate) mod tests { lifecycle.next_operation_id = 2; lifecycle.slot = SessionOperationSlot::Active(entry); } + let runtime = SessionRuntime::new(Arc::clone(&state)); registry.insert(Arc::clone(&state)); ( - Transaction::new(engine.downgrade(), key, trx_id, sts), + Transaction::new(runtime.downgrade(), key, trx_id, sts), state, ) } @@ -3645,16 +3815,17 @@ pub(crate) mod tests { )) } - fn test_session_runtime(session: &Session) -> LifecycleResult<(EngineRef, Arc)> { + fn test_session_runtime(session: &Session) -> LifecycleResult { let pin = session.pin_inspection()?; - inspect_session_in_trx(&pin.state).attach_with(|| format!("session_id={}", session.id))?; - Ok((pin.engine.clone(), Arc::clone(&pin.state))) + inspect_session_in_trx(pin.runtime.state()) + .attach_with(|| format!("session_id={}", session.id))?; + Ok(pin.runtime.clone()) } pub(crate) trait SessionTestExt { fn in_trx(&self) -> Result; fn pool_guards(&self) -> PoolGuards; - fn engine(&self) -> EngineRef; + fn engine(&self) -> SessionRuntime; fn last_cts(&self) -> TrxID; fn load_active_insert_page(&mut self, table_id: TableID) -> Option; fn save_active_insert_page(&mut self, table_id: TableID, page_id: VersionedPageID); @@ -3664,10 +3835,10 @@ pub(crate) mod tests { #[inline] fn in_trx(&self) -> Result { const OPERATION: &str = "test_inspect_transaction_state"; - let (_, state) = test_session_runtime(self) + let runtime = test_session_runtime(self) .attach_with(|| format!("operation={OPERATION}")) .disclose()?; - inspect_session_in_trx(&state) + inspect_session_in_trx(runtime.state()) .attach_with(|| format!("operation={OPERATION}, session_id={}", self.id)) .disclose() } @@ -3676,29 +3847,26 @@ pub(crate) mod tests { fn pool_guards(&self) -> PoolGuards { test_session_runtime(self) .expect("test session must be running") - .1 .pool_guards() .clone() } #[inline] - fn engine(&self) -> EngineRef { - test_session_runtime(self) - .expect("test session must be running") - .0 + fn engine(&self) -> SessionRuntime { + test_session_runtime(self).expect("test session must be running") } #[inline] fn last_cts(&self) -> TrxID { - let (_, state) = test_session_runtime(self).expect("test session must be running"); - TrxID::new(state.last_cts.load(Ordering::SeqCst)) + let runtime = test_session_runtime(self).expect("test session must be running"); + TrxID::new(runtime.state().last_cts.load(Ordering::SeqCst)) } #[inline] fn load_active_insert_page(&mut self, table_id: TableID) -> Option { test_session_runtime(self) .expect("test session must be running") - .1 + .state() .load_active_insert_page(table_id) } @@ -3706,7 +3874,7 @@ pub(crate) mod tests { fn save_active_insert_page(&mut self, table_id: TableID, page_id: VersionedPageID) { test_session_runtime(self) .expect("test session must be running") - .1 + .state() .save_active_insert_page(table_id, page_id); } } @@ -3740,7 +3908,7 @@ pub(crate) mod tests { let session_id = session.id(); let trx = session.begin_trx().unwrap(); let trx_id = trx.trx_id(); - let table_ids_before = engine.catalog().list_user_table_ids_now(); + let table_ids_before = engine.inner().core.catalog().list_user_table_ids_now(); macro_rules! assert_rejected { ($result:expr) => { @@ -3829,7 +3997,10 @@ pub(crate) mod tests { lock_entry_count(&engine, LockOwner::session_explicit(session_id)), 0 ); - assert_eq!(engine.catalog().list_user_table_ids_now(), table_ids_before); + assert_eq!( + engine.inner().core.catalog().list_user_table_ids_now(), + table_ids_before + ); trx.rollback().await.unwrap(); assert_eq!(session.list_table_ids().unwrap(), table_ids_before); @@ -3926,7 +4097,7 @@ pub(crate) mod tests { let mut session = engine.new_session().unwrap(); let session_id = session.id(); let observer = session.pin_observer().unwrap(); - let state = Arc::clone(&observer.state); + let state = Arc::clone(observer.runtime.state()); session.close().await.unwrap(); assert_eq!(session_registry_len(&engine.inner().session_registry), 1); @@ -3964,7 +4135,7 @@ pub(crate) mod tests { let session_id = session.id(); let trx = session.begin_trx().unwrap(); let observer = session.pin_observer().unwrap(); - let state = Arc::clone(&observer.state); + let state = Arc::clone(observer.runtime.state()); drop(session); { @@ -4003,7 +4174,7 @@ pub(crate) mod tests { let mut session = engine.new_session().unwrap(); let trx = session.begin_trx().unwrap(); let observer = session.pin_observer().unwrap(); - let state = Arc::clone(&observer.state); + let state = Arc::clone(observer.runtime.state()); let blocker = state.shutdown_blocker().unwrap(); assert_eq!(blocker.label(), "operation"); @@ -4123,6 +4294,48 @@ pub(crate) mod tests { }); } + #[test] + fn test_admitted_session_ref_holds_admission_until_drop() { + smol::block_on(async { + let root = TempDir::new().unwrap(); + let engine = Engine::bootstrap(EngineConfig::default().storage_root(root.path())) + .await + .unwrap(); + let session = engine.new_session().unwrap(); + let admitted = session + .session + .acquire_admission() + .unwrap() + .upgrade() + .unwrap(); + assert_eq!(admitted.runtime().state().id(), session.id()); + let (started_tx, started_rx) = mpsc::channel(); + let (done_tx, done_rx) = mpsc::channel(); + + thread::scope(|scope| { + let shutdown = scope.spawn(|| { + started_tx.send(()).unwrap(); + engine.shutdown(); + done_tx.send(()).unwrap(); + }); + + started_rx + .recv_timeout(Duration::from_secs(5)) + .expect("shutdown thread should start"); + assert!( + done_rx.recv_timeout(Duration::from_millis(20)).is_err(), + "shutdown must wait while the admitted session reference is live" + ); + + drop(admitted); + done_rx + .recv_timeout(Duration::from_secs(5)) + .expect("shutdown should complete after admitted session reference drops"); + shutdown.join().unwrap(); + }); + }); + } + #[test] fn test_normal_observer_admission_race_registers_or_rejects() { run_observer_admission_shutdown_race(false); @@ -4155,24 +4368,27 @@ pub(crate) mod tests { let pin = session.pin_observer().unwrap(); let cached_page = { - let cache = pin.state.table_cache.lock(); + let cache = pin.runtime.state().table_cache.lock(); let entry = cache.get(&table_id).unwrap(); assert!(entry.table.upgrade().is_some()); entry.active_insert_page.unwrap() }; assert_eq!( - pin.state.load_active_insert_page(table_id), + pin.runtime.state().load_active_insert_page(table_id), Some(cached_page) ); { - let cache = pin.state.table_cache.lock(); + let cache = pin.runtime.state().table_cache.lock(); let entry = cache.get(&table_id).unwrap(); assert!(entry.table.upgrade().is_some()); assert!(entry.active_insert_page.is_none()); } - pin.state.save_active_insert_page(table_id, cached_page); + pin.runtime + .state() + .save_active_insert_page(table_id, cached_page); assert_eq!( - pin.state + pin.runtime + .state() .table_cache .lock() .get(&table_id) @@ -4190,6 +4406,8 @@ pub(crate) mod tests { .await .unwrap(); let catalog_tables = engine + .inner() + .core .catalog() .storage .get_catalog_table(TABLE_ID_TABLES) @@ -4203,7 +4421,8 @@ pub(crate) mod tests { { let pin = session1.pin_observer().unwrap(); assert!( - pin.state + pin.runtime + .state() .table_cache .lock() .keys() @@ -4220,7 +4439,8 @@ pub(crate) mod tests { ); let pin = session2.pin_observer().unwrap(); assert!( - pin.state + pin.runtime + .state() .table_cache .lock() .keys() @@ -4238,7 +4458,7 @@ pub(crate) mod tests { .unwrap(); let session_id = SessionID::new(1); let trx_id = MIN_ACTIVE_TRX_ID; - let state = SessionState::new(engine.new_ref().unwrap(), session_id); + let state = Arc::new(new_session_state_for_test(&engine, session_id)); let key = SessionOperationKey::new(session_id, OperationID::new(1)); let entry = SessionOperationEntry::new_public_transaction( key, @@ -4254,8 +4474,12 @@ pub(crate) mod tests { let blocker = state .shutdown_blocker() .expect("abandoned transaction must block shutdown"); - assert_eq!(blocker.cleanup(), Some((key, trx_id))); assert_eq!(blocker.label(), "operation"); + let cleanup = blocker + .into_cleanup() + .expect("abandoned transaction must be claimable"); + assert_eq!((cleanup.operation_key, cleanup.trx_id), (key, trx_id)); + assert!(Arc::ptr_eq(cleanup.runtime.state(), &state)); assert_eq!(entry.inspect().state, SessionOperationState::CleanupReady); }); } @@ -4269,7 +4493,7 @@ pub(crate) mod tests { .unwrap(); let session_id = SessionID::new(1); let trx_id = MIN_ACTIVE_TRX_ID; - let state = SessionState::new(engine.new_ref().unwrap(), session_id); + let state = Arc::new(new_session_state_for_test(&engine, session_id)); let key = SessionOperationKey::new(session_id, OperationID::new(1)); let entry = SessionOperationEntry::new_public_transaction( key, @@ -4285,11 +4509,11 @@ pub(crate) mod tests { let blocker = state .shutdown_blocker() .expect("failed-retained operation must block shutdown"); - assert_eq!(blocker.cleanup(), None); assert_eq!( blocker.operation_state(), Some(SessionOperationState::FailedRetained) ); + assert!(blocker.into_cleanup().is_none()); assert_eq!(entry.inspect().state, SessionOperationState::FailedRetained); }); } @@ -4303,7 +4527,7 @@ pub(crate) mod tests { .unwrap(); let session_id = SessionID::new(1); let trx_id = MIN_ACTIVE_TRX_ID; - let state = SessionState::new(engine.new_ref().unwrap(), session_id); + let state = Arc::new(new_session_state_for_test(&engine, session_id)); let key = SessionOperationKey::new(session_id, OperationID::new(1)); let entry = SessionOperationEntry::new_public_transaction( key, @@ -4314,11 +4538,12 @@ pub(crate) mod tests { let shutdown_wait = state .shutdown_wait() .expect("active transaction must install a shutdown listener"); - assert_eq!(shutdown_wait.blocker.cleanup(), None); + let SessionShutdownWait { blocker, listener } = shutdown_wait; + assert!(blocker.into_cleanup().is_none()); assert!(state.lifecycle.lock().change_ev.is_some()); assert!(state.abandon_trx_handle(key, trx_id)); - shutdown_wait.listener.await; + listener.await; assert_eq!(entry.inspect().state, SessionOperationState::CleanupReady); }); } @@ -4332,7 +4557,7 @@ pub(crate) mod tests { .unwrap(); let session_id = SessionID::new(1); let trx_id = MIN_ACTIVE_TRX_ID; - let state = SessionState::new(engine.new_ref().unwrap(), session_id); + let state = Arc::new(new_session_state_for_test(&engine, session_id)); let key = SessionOperationKey::new(session_id, OperationID::new(1)); let entry = SessionOperationEntry::new_public_transaction( key, @@ -4365,7 +4590,7 @@ pub(crate) mod tests { for raw_id in 1..=2 { let session_id = SessionID::new(raw_id); let trx_id = TrxID::new(MIN_ACTIVE_TRX_ID.as_u64() + raw_id); - let state = Arc::new(SessionState::new(engine.new_ref().unwrap(), session_id)); + let state = Arc::new(new_session_state_for_test(&engine, session_id)); let key = SessionOperationKey::new(session_id, OperationID::new(1)); let entry = SessionOperationEntry::new_public_transaction( key, @@ -4385,14 +4610,11 @@ pub(crate) mod tests { let shutdown_wait = registry .first_shutdown_wait() .expect("one active session must block shutdown"); - assert!( - expected_cleanup.contains( - &shutdown_wait - .blocker - .cleanup() - .expect("abandoned transaction must be claimable") - ) - ); + let cleanup = shutdown_wait + .blocker + .into_cleanup() + .expect("abandoned transaction must be claimable"); + assert!(expected_cleanup.contains(&(cleanup.operation_key, cleanup.trx_id))); assert_eq!( states .iter() @@ -4404,6 +4626,45 @@ pub(crate) mod tests { }); } + #[test] + fn test_cold_removal_preserves_pointer_distinct_replacement() { + smol::block_on(async { + let root = TempDir::new().unwrap(); + let engine = Engine::bootstrap(EngineConfig::default().storage_root(root.path())) + .await + .unwrap(); + let session = engine.new_session().unwrap(); + let session_id = session.id(); + let registry = &engine.inner().session_registry; + let stale = registry + .session_state(session_id) + .expect("new session must be registered"); + let stale_runtime = SessionRuntime::new(Arc::clone(&stale)); + let replacement = Arc::new(new_session_state_for_test(&engine, session_id)); + + let displaced = registry + .entries + .insert(session_id, Arc::clone(&replacement)) + .expect("test replacement must displace the original state"); + assert!(Arc::ptr_eq(&displaced, &stale)); + + stale_runtime.remove_if_requested(true); + let registered = registry + .session_state(session_id) + .expect("pointer-distinct replacement must remain registered"); + assert!(Arc::ptr_eq(®istered, &replacement)); + + remove_session_for_test(registry, session_id); + drop(registered); + drop(replacement); + drop(displaced); + drop(stale_runtime); + drop(stale); + drop(session); + engine.shutdown(); + }); + } + #[test] fn test_registry_shutdown_wait_lazily_drains_many_blockers() { smol::block_on(async { @@ -4418,7 +4679,7 @@ pub(crate) mod tests { for raw_id in 1..=SESSION_COUNT { let session_id = SessionID::new(raw_id); let trx_id = TrxID::new(MIN_ACTIVE_TRX_ID.as_u64() + raw_id); - let state = Arc::new(SessionState::new(engine.new_ref().unwrap(), session_id)); + let state = Arc::new(new_session_state_for_test(&engine, session_id)); let key = SessionOperationKey::new(session_id, OperationID::new(1)); let entry = SessionOperationEntry::new_public_transaction( key, @@ -4439,10 +4700,11 @@ pub(crate) mod tests { let shutdown_wait = registry .first_shutdown_wait() .expect("one remaining active session must block shutdown"); - let (key, _) = shutdown_wait + let cleanup = shutdown_wait .blocker - .cleanup() + .into_cleanup() .expect("abandoned transaction must be claimable"); + let key = cleanup.operation_key; assert!( !drained.contains(&key), "a drained session must not block a later lazy pass" @@ -4673,7 +4935,7 @@ pub(crate) mod tests { .unwrap(); let key = operation.key(); let entry = Arc::clone(&operation.entry); - let state = Arc::clone(&operation.state); + let state = Arc::clone(operation.runtime.state()); assert!(state.lifecycle.lock().change_ev.is_none()); let public_cache_ptr = state .lifecycle @@ -5065,7 +5327,15 @@ pub(crate) mod tests { let engine = Engine::bootstrap(EngineConfig::default().storage_root(&main_dir)) .await .unwrap(); - assert!(engine.catalog().get_table(table_id).await.is_some()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_some() + ); assert!( engine .inner() @@ -5133,7 +5403,7 @@ pub(crate) mod tests { assert_freeze_created(session.freeze_table(table_id, usize::MAX).await.unwrap()); assert_checkpoint_published(&mut session, table_id).await; - let before = engine.catalog().storage.checkpoint_snapshot(); + let before = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(before.meta.first_redo_log_seq, 0); let outcome = session @@ -5162,7 +5432,7 @@ pub(crate) mod tests { ); assert_eq!(outcome.redo_truncation.failed_unlink_files, 0); - let after = engine.catalog().storage.checkpoint_snapshot(); + let after = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(after.catalog_replay_start_ts, catalog_replay_start_ts); assert_eq!( after.meta.first_redo_log_seq, @@ -5204,7 +5474,7 @@ pub(crate) mod tests { commit_redo_durability_anchor(&mut session, table_id).await; session.checkpoint_catalog().await.unwrap(); - let checkpointed = engine.catalog().storage.checkpoint_snapshot(); + let checkpointed = engine.inner().core.catalog().storage.checkpoint_snapshot(); let outcome = session .checkpoint_catalog_and_truncate_redo_log() .await @@ -5216,7 +5486,7 @@ pub(crate) mod tests { outcome.redo_truncation.new_first_retained_file_seq > 0, "{outcome:?}" ); - let after = engine.catalog().storage.checkpoint_snapshot(); + let after = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!( after.catalog_replay_start_ts, checkpointed.catalog_replay_start_ts @@ -5238,7 +5508,13 @@ pub(crate) mod tests { .await .unwrap(); let table_id = create_rotated_redo_table(&engine, &main_dir, log_file_stem, 2).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let root_floor = table.redo_replay_floor_snapshot(); drop(table); let mut session = engine.new_session().unwrap(); @@ -5253,6 +5529,8 @@ pub(crate) mod tests { "{checkpoint:?}" ); let watermark = engine + .inner() + .core .catalog() .storage .table_replay_silent_watermarks() @@ -5264,6 +5542,8 @@ pub(crate) mod tests { assert!(watermark.deletion_cutoff_ts > root_floor.deletion_cutoff_ts); assert!( engine + .inner() + .core .catalog() .storage .checkpointed_silent_watermarks() @@ -5313,7 +5593,12 @@ pub(crate) mod tests { )), "{outcome:?}" ); - let checkpointed = engine.catalog().storage.checkpointed_silent_watermarks(); + let checkpointed = engine + .inner() + .core + .catalog() + .storage + .checkpointed_silent_watermarks(); let checkpointed_floor = checkpointed .get(&table_id) .copied() @@ -5344,7 +5629,7 @@ pub(crate) mod tests { assert_freeze_created(session.freeze_table(table_id, usize::MAX).await.unwrap()); assert_checkpoint_published(&mut session, table_id).await; - let before = engine.catalog().storage.checkpoint_snapshot(); + let before = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(before.meta.first_redo_log_seq, 0); let plan = engine.inner().trx_sys.plan_redo_truncation().unwrap(); assert_eq!(plan.first_retained_file_seq, 0); @@ -5386,7 +5671,7 @@ pub(crate) mod tests { Some(FatalError::CheckpointWrite) ); assert!(publish_hook.call_count() > 0); - let after = engine.catalog().storage.checkpoint_snapshot(); + let after = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!( after.catalog_replay_start_ts, before.catalog_replay_start_ts @@ -5418,7 +5703,7 @@ pub(crate) mod tests { commit_redo_durability_anchor(&mut session, table_id).await; session.checkpoint_catalog().await.unwrap(); - let before = engine.catalog().storage.checkpoint_snapshot(); + let before = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!(before.meta.first_redo_log_seq, 0); let plan = engine.inner().trx_sys.plan_redo_truncation().unwrap(); assert_eq!(plan.first_retained_file_seq, 0); @@ -5460,7 +5745,7 @@ pub(crate) mod tests { Some(FatalError::CheckpointWrite) ); assert!(publish_hook.call_count() > 0); - let after = engine.catalog().storage.checkpoint_snapshot(); + let after = engine.inner().core.catalog().storage.checkpoint_snapshot(); assert_eq!( after.catalog_replay_start_ts, before.catalog_replay_start_ts @@ -5489,6 +5774,8 @@ pub(crate) mod tests { let mut setup_session = engine.new_session().unwrap(); setup_session.checkpoint_catalog().await.unwrap(); engine + .inner() + .core .catalog() .storage .publish_first_redo_log_seq(1) @@ -5499,7 +5786,7 @@ pub(crate) mod tests { let hook_called = Arc::new(AtomicBool::new(false)); let hook_flag = Arc::clone(&hook_called); - let hook_engine = engine.new_ref().unwrap(); + let hook_catalog = engine.inner().core.catalog.clone(); let hook_guard = install_redo_cleanup_before_unlink_hook( &engine.inner().maintenance_test, Arc::new(move |file_seq, _path| { @@ -5507,7 +5794,7 @@ pub(crate) mod tests { return; } hook_flag.store(true, Ordering::SeqCst); - let catalog = hook_engine.catalog(); + let catalog = &*hook_catalog; let mut metadata_fut = Box::pin(catalog.acquire_index_metadata_change()); let waker = noop_waker(); let mut cx = Context::from_waker(&waker); @@ -5552,6 +5839,8 @@ pub(crate) mod tests { let mut setup_session = engine.new_session().unwrap(); setup_session.checkpoint_catalog().await.unwrap(); engine + .inner() + .core .catalog() .storage .publish_first_redo_log_seq(1) @@ -5620,6 +5909,8 @@ pub(crate) mod tests { ); assert_eq!( engine + .inner() + .core .catalog() .storage .checkpoint_snapshot() @@ -5666,13 +5957,27 @@ pub(crate) mod tests { let table_id = create_rotated_redo_table(&engine, &main_dir, log_file_stem, 1).await; let mut session = engine.new_session().unwrap(); session.checkpoint_catalog().await.unwrap(); - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let expected_floor = table.redo_replay_floor_snapshot(); drop(table); session.drop_table(table_id).await.unwrap(); - assert!(engine.catalog().get_table(table_id).await.is_none()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_none() + ); assert_eq!(session.list_table_ids().unwrap(), Vec::::new()); let plan = engine.inner().trx_sys.plan_redo_truncation().unwrap(); assert!( @@ -5699,7 +6004,7 @@ pub(crate) mod tests { let engine = Engine::bootstrap(EngineConfig::default().storage_root(root.path())) .await .unwrap(); - let catalog = engine.catalog(); + let catalog = engine.inner().core.catalog(); catalog.acquire_index_metadata_change().await; let mut session = engine.new_session().unwrap(); let mut truncate_fut = Box::pin(session.truncate_redo_log()); @@ -5855,6 +6160,8 @@ pub(crate) mod tests { assert!(publish_hook.call_count() > 0); assert_eq!( engine + .inner() + .core .catalog() .storage .checkpoint_snapshot() @@ -5883,6 +6190,8 @@ pub(crate) mod tests { .unwrap(); create_rotated_redo_table(&engine, &main_dir, log_file_stem, 1).await; engine + .inner() + .core .catalog() .storage .publish_first_redo_log_seq(1) @@ -5893,7 +6202,7 @@ pub(crate) mod tests { let hook_called = Arc::new(AtomicBool::new(false)); let hook_flag = Arc::clone(&hook_called); - let hook_engine = engine.new_ref().unwrap(); + let hook_catalog = engine.inner().core.catalog.clone(); let hook_guard = install_redo_cleanup_before_unlink_hook( &engine.inner().maintenance_test, Arc::new(move |file_seq, _path| { @@ -5901,7 +6210,7 @@ pub(crate) mod tests { return; } hook_flag.store(true, Ordering::SeqCst); - let catalog = hook_engine.catalog(); + let catalog = &*hook_catalog; let mut metadata_fut = Box::pin(catalog.acquire_index_metadata_change()); let waker = noop_waker(); let mut cx = Context::from_waker(&waker); @@ -5940,6 +6249,8 @@ pub(crate) mod tests { .unwrap(); create_rotated_redo_table(&engine, &main_dir, log_file_stem, 1).await; engine + .inner() + .core .catalog() .storage .publish_first_redo_log_seq(1) @@ -5988,6 +6299,8 @@ pub(crate) mod tests { .unwrap(); create_rotated_redo_table(&engine, &main_dir, log_file_stem, 1).await; engine + .inner() + .core .catalog() .storage .publish_first_redo_log_seq(1) @@ -6036,6 +6349,8 @@ pub(crate) mod tests { .unwrap(); create_rotated_redo_table(&engine, &main_dir, log_file_stem, 1).await; engine + .inner() + .core .catalog() .storage .publish_first_redo_log_seq(1) diff --git a/doradb-storage/src/table/access.rs b/doradb-storage/src/table/access.rs index df7dbe31..58eabd50 100644 --- a/doradb-storage/src/table/access.rs +++ b/doradb-storage/src/table/access.rs @@ -753,9 +753,11 @@ impl<'op> UserTableAccessor<'op> { let root = self .storage .with_active_root(&proof, |root| root.secondary_index_roots[index_no]); + let pool_guards = rt.pool_guards(); Ok(OwnedCurrentIndexReadHandle::new( index, - rt.pool_guards().clone(), + pool_guards.index_guard().clone(), + pool_guards.disk_guard().clone(), root, &proof, transaction, @@ -5491,7 +5493,7 @@ mod tests { .await; trx.commit().await.unwrap(); - let allocated_after_route = engine.inner().disk_pool.allocated(); + let allocated_after_route = engine.inner().pools.disk.allocated(); assert!(allocated_after_route >= 1); expect_select_committed(table_id, &mut session, &key, |vals| { @@ -5499,7 +5501,7 @@ mod tests { assert_eq!(vals[1], Val::from("name")); }) .await; - let allocated_after_first = engine.inner().disk_pool.allocated(); + let allocated_after_first = engine.inner().pools.disk.allocated(); assert!(allocated_after_first >= allocated_after_route); expect_select_committed(table_id, &mut session, &key, |vals| { @@ -5507,7 +5509,7 @@ mod tests { assert_eq!(vals[1], Val::from("name")); }) .await; - assert_eq!(engine.inner().disk_pool.allocated(), allocated_after_first); + assert_eq!(engine.inner().pools.disk.allocated(), allocated_after_first); }); } @@ -6748,7 +6750,8 @@ mod tests { }; let page_guard = engine .inner() - .mem_pool + .pools + .mem .get_page::( session.pool_guards().mem_guard(), page_id, @@ -6764,7 +6767,8 @@ mod tests { let insert_page_guard = engine .inner() - .mem_pool + .pools + .mem .get_page::( session.pool_guards().mem_guard(), page_id, @@ -6840,7 +6844,8 @@ mod tests { let mut trx = session.begin_trx().unwrap(); let page_guard = engine .inner() - .mem_pool + .pools + .mem .get_page::( session.pool_guards().mem_guard(), page_id, @@ -8841,6 +8846,8 @@ mod tests { let resource = LockResource::TableMetadata(table_id); let blocker = LockOwner::session_explicit(SessionID::new(91_225)); engine + .inner() + .core .lock_manager() .acquire(resource, LockMode::Exclusive, blocker) .await @@ -8866,7 +8873,10 @@ mod tests { LockDebugEntryState::Waiting, ) .await; - assert_eq!(engine.lock_manager().release_owner(stmt_owner), 1); + assert_eq!( + engine.inner().core.lock_manager().release_owner(stmt_owner), + 1 + ); let err = scan_fut.await.unwrap_err(); assert_eq!( @@ -8878,7 +8888,14 @@ mod tests { assert_eq!(rendered.matches(&format!("table_id={table_id}")).count(), 1); assert!(rendered.contains("resource=table_metadata"), "{rendered}"); assert!(rendered.contains("mode=shared"), "{rendered}"); - assert_eq!(engine.lock_manager().release(resource, blocker), 1); + assert_eq!( + engine + .inner() + .core + .lock_manager() + .release(resource, blocker), + 1 + ); trx.rollback().await.unwrap(); }); } @@ -9179,7 +9196,8 @@ mod tests { .exec(async |stmt| { let page_guard = engine .inner() - .mem_pool + .pools + .mem .get_page::( session.pool_guards().mem_guard(), page_id, @@ -10031,7 +10049,7 @@ mod tests { inserted.push((row_id, key)); } trx.commit().await.unwrap(); - let stats = engine.inner().index_pool.stats(); + let stats = engine.inner().pools.index.stats(); if stats.completed_writes > 0 && stats.write_errors == 0 { break; } @@ -10039,14 +10057,14 @@ mod tests { // Timer audit: index-pool eviction/I/O test coordination. for _ in 0..20 { - let stats = engine.inner().index_pool.stats(); + let stats = engine.inner().pools.index.stats(); if stats.completed_writes > 0 && stats.write_errors == 0 { break; } Timer::after(Duration::from_millis(50)).await; } - let stats = engine.inner().index_pool.stats(); + let stats = engine.inner().pools.index.stats(); assert!( stats.completed_writes > 0 && stats.write_errors == 0, "user secondary-index pool should evict with a small index buffer" diff --git a/doradb-storage/src/table/gc.rs b/doradb-storage/src/table/gc.rs index 18dbb5fb..89a2f731 100644 --- a/doradb-storage/src/table/gc.rs +++ b/doradb-storage/src/table/gc.rs @@ -277,7 +277,7 @@ async fn execute_mem_index_cleanup_inner( ) -> RuntimeOrFatalResult { let table = Arc::clone(&resources.table); let clean_live_entries = resources.clean_live_entries; - let trx_sys = scope.engine().trx_sys.clone(); + let trx_sys = &scope.engine().trx_sys; let pool_guards = scope.pool_guards(); loop { let trx = scope @@ -328,7 +328,7 @@ async fn execute_mem_index_cleanup_inner( } else { let cleanup_res = table .cleanup_secondary_mem_indexes_at_snapshot( - &pool_guards, + pool_guards, &snapshot, clean_live_entries, ) @@ -971,9 +971,8 @@ mod tests { let reader_sts = Arc::new(parking_lot::Mutex::new(TrxID::new(0))); let hook_reader_holder = Arc::clone(&reader_holder); let hook_reader_sts = Arc::clone(&reader_sts); - let hook_engine = engine.new_ref().unwrap(); + let mut reader_session = engine.new_session().unwrap(); set_test_checkpoint_after_trx_start_hook(&engine, move || async move { - let mut reader_session = hook_engine.new_session().unwrap(); let reader = reader_session.begin_trx().unwrap(); *hook_reader_sts.lock() = reader.sts(); *hook_reader_holder.lock() = Some((reader_session, reader)); diff --git a/doradb-storage/src/table/layout.rs b/doradb-storage/src/table/layout.rs index b8194372..ae265d7d 100644 --- a/doradb-storage/src/table/layout.rs +++ b/doradb-storage/src/table/layout.rs @@ -335,11 +335,15 @@ mod tests { ); let current_cts = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() .effective_cts(); let installed = engine + .inner() + .core .catalog() .install_index_layout_and_publish_history( table_id, @@ -374,7 +378,7 @@ mod tests { ); let guards = PoolGuards::builder() - .push(PoolRole::Index, engine.inner().index_pool.pool_guard()) + .push(PoolRole::Index, engine.inner().pools.index.pool_guard()) .build(); assert_eq!( table_for_internal_assertion(&engine, table_id) diff --git a/doradb-storage/src/table/mem_table.rs b/doradb-storage/src/table/mem_table.rs index 9ef8c518..39fa38f4 100644 --- a/doradb-storage/src/table/mem_table.rs +++ b/doradb-storage/src/table/mem_table.rs @@ -3386,11 +3386,11 @@ mod tests { mem_table_id: TableID, metadata: Arc, ) -> TestMemTable { - let meta_guard = engine.inner().meta_pool.pool_guard(); - let index_guard = engine.inner().index_pool.pool_guard(); - let mem_pool = engine.inner().mem_pool.clone_inner(); + let meta_guard = engine.inner().pools.meta.pool_guard(); + let index_guard = engine.inner().pools.index.pool_guard(); + let mem_pool = engine.inner().pools.mem.clone(); let blk_idx = BlockIndex::new( - engine.inner().meta_pool.clone_inner(), + engine.inner().pools.meta.clone(), &meta_guard, RowID::new(0), SUPER_BLOCK_ID, @@ -3400,7 +3400,7 @@ mod tests { MemTable::new( mem_pool.clone(), mem_pool.row_pool_role(), - engine.inner().index_pool.clone_inner(), + engine.inner().pools.index.clone(), PoolRole::Index, &index_guard, mem_table_id, diff --git a/doradb-storage/src/table/mod.rs b/doradb-storage/src/table/mod.rs index c2059972..96ebda0f 100644 --- a/doradb-storage/src/table/mod.rs +++ b/doradb-storage/src/table/mod.rs @@ -2373,6 +2373,8 @@ pub(crate) mod tests { #[inline] pub(crate) fn table_for_internal_assertion(engine: &Engine, table_id: TableID) -> Arc
{ engine + .inner() + .core .catalog() .get_table_now(table_id) .expect("test table should exist") @@ -2587,7 +2589,7 @@ pub(crate) mod tests { } pub(crate) fn lock_entry_count(engine: &Engine, owner: LockOwner) -> usize { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .filter(|entry| entry.owner == owner) @@ -2601,7 +2603,7 @@ pub(crate) mod tests { mode: LockMode, state: LockDebugEntryState, ) -> bool { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .any(|entry| { @@ -2617,7 +2619,7 @@ pub(crate) mod tests { owner: LockOwner, resource: LockResource, ) -> bool { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .any(|entry| entry.owner == owner && entry.resource == resource) @@ -2628,7 +2630,7 @@ pub(crate) mod tests { session_id: SessionID, resource: LockResource, ) -> Option { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .find(|entry| { @@ -2654,7 +2656,7 @@ pub(crate) mod tests { mode: LockMode, state: LockDebugEntryState, ) -> Option { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .find(|entry| { diff --git a/doradb-storage/src/table/persistence.rs b/doradb-storage/src/table/persistence.rs index c35db113..0bb7b778 100644 --- a/doradb-storage/src/table/persistence.rs +++ b/doradb-storage/src/table/persistence.rs @@ -226,8 +226,8 @@ where let table_id = table.table_id(); let table_file = table.file(); let disk_pool = table.disk_pool(); - let trx_sys = session.engine().trx_sys.clone(); - let table_writes = session.engine().table_fs.background_writes().clone(); + let trx_sys = &session.engine().trx_sys; + let table_writes = session.engine().table_fs.background_writes(); let pool_guards = session.pool_guards(); if let Some(reason) = table.active_root_checkpoint_delay(session) { return Ok(CheckpointOutcome::Delayed { reason }); @@ -238,7 +238,7 @@ where // Step 1: claim one mutable root snapshot and initialize checkpoint // boundaries. This is checkpoint-internal current-root access after the // post-lease liveness check above. - let mut mutable_file = MutableTableFile::fork(table_file, &table_writes, disk_pool.clone()); + let mut mutable_file = MutableTableFile::fork(table_file, table_writes, disk_pool.clone()); let pivot_row_id = mutable_file.root().pivot_row_id; let mut secondary_sidecar = SecondaryCheckpointSidecar::new(metadata); @@ -271,7 +271,7 @@ where Some(heap_redo_start_ts) => Some(heap_redo_start_ts), None => { let heap_redo_start_ts = table - .heap_redo_start_from(&pool_guards, heap_redo_start_row_id) + .heap_redo_start_from(pool_guards, heap_redo_start_row_id) .await .change_context(RuntimeError::CheckpointExecution) .attach_with(|| { @@ -287,7 +287,7 @@ where }; if !pages.is_empty() { let transition_pages = table - .load_frozen_pages_for_transition(&pool_guards, &pages) + .load_frozen_pages_for_transition(pool_guards, &pages) .await .change_context(RuntimeError::CheckpointExecution) .attach_with(|| { @@ -362,7 +362,7 @@ where let mut lwc_blocks = table .build_lwc_blocks( metadata, - &pool_guards, + pool_guards, self.attempt .batch() .map(|batch| batch.prepared.as_slice()) @@ -484,7 +484,7 @@ where ) })?; sys_trx - .upsert_silent_watermark(session.engine().catalog(), &pool_guards, watermark) + .upsert_silent_watermark(session.engine().catalog(), pool_guards, watermark) .await .change_context(RuntimeError::CheckpointExecution) .attach_with(|| { @@ -1677,7 +1677,7 @@ impl Table { where S: SessionRuntimeAccess + ?Sized, { - let engine = session.engine(); + let engine = session.runtime(); let trx_sys = &engine.trx_sys; ensure_maintenance_wait_running(session, "observe active-root checkpoint retry")?; if self.active_root_retry_ready(effective_ts, trx_sys.published_gc_horizon()) { @@ -1734,7 +1734,7 @@ impl Table { let Some(page_idx) = batch.pages.iter().position(|page| page.page_id == page_id) else { return Ok(CheckpointRetryObservation::Ready); }; - let engine = session.engine(); + let engine = session.runtime(); let trx_sys = &engine.trx_sys; loop { @@ -1857,7 +1857,7 @@ impl Table { }); let guards = session.pool_guards(); let mut page_guards = self - .load_frozen_pages_for_transition(&guards, &[page_info]) + .load_frozen_pages_for_transition(guards, &[page_info]) .await?; let page_guard = page_guards.pop().unwrap_or_else(|| { panic!( @@ -1900,7 +1900,7 @@ impl Table { let mut pages = Vec::new(); let mut reached_row_budget = false; let mut heap_redo_start_ts = None; - self.mem_scan(&guards, |page_guard| { + self.mem_scan(guards, |page_guard| { if reached_row_budget { heap_redo_start_ts = Some(page_guard.unwrap_vmap().create_cts()); return false; @@ -1917,7 +1917,7 @@ impl Table { }) .await?; let page_guards = self - .load_frozen_pages_for_transition(&guards, &pages) + .load_frozen_pages_for_transition(guards, &pages) .await?; #[cfg(test)] test_hooks::run_test_freeze_after_loading_hook(&session.engine().maintenance_test).await; @@ -2183,7 +2183,7 @@ fn ensure_maintenance_wait_running( where S: SessionRuntimeAccess + ?Sized, { - let engine = session.engine(); + let engine = session.runtime(); engine.poisoner.ensure_healthy()?; if engine.shutdown_started() { return Err(RuntimeOrFatalError::from( @@ -2429,7 +2429,7 @@ mod tests { thread::scope(|scope| { let shutdown = scope.spawn(|| engine.shutdown()); - while !pin.engine.shutdown_started() { + while !pin.runtime.shutdown_started() { thread::yield_now(); } @@ -2622,7 +2622,14 @@ mod tests { redo_cts }; assert!(checkpoint_redo_cts < drop_session.last_cts()); - assert!(engine.catalog().get_table_now(table_id).is_none()); + assert!( + engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .is_none() + ); } #[test] @@ -3530,6 +3537,8 @@ mod tests { let root_after = table.file().active_root_unchecked().clone(); if root_after.deletion_cutoff_ts <= root_before.deletion_cutoff_ts { let watermark = engine + .inner() + .core .catalog() .storage .table_replay_silent_watermarks() @@ -3541,10 +3550,14 @@ mod tests { insert_rows(table_id, &mut session, 1, 2, "durability-anchor").await; session.checkpoint_catalog().await.unwrap(); } - let effective = engine.catalog().effective_user_table_redo_replay_floor( - table_id, - table.redo_replay_floor_snapshot(), - ); + let effective = engine + .inner() + .core + .catalog() + .effective_user_table_redo_replay_floor( + table_id, + table.redo_replay_floor_snapshot(), + ); assert!( effective.deletion_cutoff_ts > root_before.deletion_cutoff_ts, "{effective:?}" @@ -3884,6 +3897,8 @@ mod tests { let checkpoint_ts = assert_checkpoint_published(&mut session, table_id).await; let watermark = engine + .inner() + .core .catalog() .storage .table_replay_silent_watermarks() @@ -4466,9 +4481,8 @@ mod tests { let reader_sts = Arc::new(parking_lot::Mutex::new(TrxID::new(0))); let hook_reader_holder = Arc::clone(&reader_holder); let hook_reader_sts = Arc::clone(&reader_sts); - let hook_engine = engine.new_ref().unwrap(); + let mut reader_session = engine.new_session().unwrap(); set_test_checkpoint_after_trx_start_hook(&engine, move || async move { - let mut reader_session = hook_engine.new_session().unwrap(); let reader = reader_session.begin_trx().unwrap(); *hook_reader_sts.lock() = reader.sts(); *hook_reader_holder.lock() = Some((reader_session, reader)); @@ -5035,7 +5049,14 @@ mod tests { assert_freeze_created(freeze.await.unwrap()); drop(table); drop_table.await.unwrap(); - assert!(engine.catalog().get_table_now(table_id).is_none()); + assert!( + engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .is_none() + ); }); } @@ -5278,7 +5299,14 @@ mod tests { redo_cts }; assert!(checkpoint_redo_cts < drop_session.last_cts()); - assert!(engine.catalog().get_table_now(table_id).is_none()); + assert!( + engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .is_none() + ); }); } @@ -6025,6 +6053,8 @@ mod tests { evictable_test_engine(&temp_dir, 64u64 * 1024 * 1024, "redo_testsys").await; let table_id = create_table2_for_test(&engine).await; let table = engine + .inner() + .core .catalog() .get_table_now(table_id) .expect("test table should exist"); @@ -6046,7 +6076,7 @@ mod tests { let table_file = engine .inner() .table_fs - .open_table_file(table_id, engine.inner().disk_pool.clone_inner()) + .open_table_file(table_id, engine.inner().pools.disk.clone()) .await .unwrap(); let root_after = table_file.active_root_unchecked(); @@ -6097,6 +6127,8 @@ mod tests { table_id: TableID, ) { let watermark = engine + .inner() + .core .catalog() .storage .table_replay_silent_watermarks() @@ -6139,6 +6171,8 @@ mod tests { "Idle" ); let watermark = engine + .inner() + .core .catalog() .storage .table_replay_silent_watermarks() @@ -6278,6 +6312,8 @@ mod tests { let guards = session.pool_guards(); let watermark = engine + .inner() + .core .catalog() .storage .table_replay_silent_watermarks() @@ -6290,6 +6326,8 @@ mod tests { assert!(watermark.deletion_cutoff_ts > root_before.deletion_cutoff_ts); assert!( engine + .inner() + .core .catalog() .storage .checkpointed_silent_watermarks() @@ -6298,8 +6336,10 @@ mod tests { "uncheckpointed watermark rows must not update durable cache" ); - let snapshot = engine.catalog().storage.checkpoint_snapshot(); + let snapshot = engine.inner().core.catalog().storage.checkpoint_snapshot(); let (live_before_catalog_checkpoint, _) = engine + .inner() + .core .catalog() .snapshot_user_table_redo_floors(snapshot.catalog_replay_start_ts); assert_eq!(live_before_catalog_checkpoint.len(), 1); @@ -6310,7 +6350,12 @@ mod tests { insert_rows(table_id, &mut session, 40, 41, &name).await; session.checkpoint_catalog().await.unwrap(); - let checkpointed = engine.catalog().storage.checkpointed_silent_watermarks(); + let checkpointed = engine + .inner() + .core + .catalog() + .storage + .checkpointed_silent_watermarks(); let checkpointed_floor = checkpointed .get(&table_id) .copied() @@ -6323,8 +6368,10 @@ mod tests { checkpointed_floor.deletion_cutoff_ts, watermark.deletion_cutoff_ts ); - let snapshot = engine.catalog().storage.checkpoint_snapshot(); + let snapshot = engine.inner().core.catalog().storage.checkpoint_snapshot(); let (live_after_catalog_checkpoint, _) = engine + .inner() + .core .catalog() .snapshot_user_table_redo_floors(snapshot.catalog_replay_start_ts); assert_eq!(live_after_catalog_checkpoint.len(), 1); @@ -6448,16 +6495,16 @@ mod tests { let name = "g".repeat(1024); insert_rows(table_id, &mut session, 0, 200, &name).await; - let allocated_before = engine.inner().mem_pool.allocated(); + let allocated_before = engine.inner().pools.mem.allocated(); assert_freeze_created(session.freeze_table(table_id, usize::MAX).await.unwrap()); let outcome = session.checkpoint_table_with_wait(table_id).await.unwrap(); let CheckpointOutcome::Published { redo_cts, .. } = outcome else { panic!("checkpoint should publish, got {outcome:?}"); }; - let allocated_after = engine.inner().mem_pool.allocated(); + let allocated_after = engine.inner().pools.mem.allocated(); wait_for_checkpoint_purge(&session, redo_cts).await; let reclaimed = allocated_after < allocated_before - || engine.inner().mem_pool.allocated() < allocated_before; + || engine.inner().pools.mem.allocated() < allocated_before; assert!(reclaimed, "row pages should be reclaimed after purge"); }); } @@ -6504,7 +6551,7 @@ mod tests { ); let retired_page_ids = table.checkpoint_workflow.frozen_page_ids().unwrap(); assert!(!retired_page_ids.is_empty()); - let allocated_before_checkpoint = engine.inner().mem_pool.allocated(); + let allocated_before_checkpoint = engine.inner().pools.mem.allocated(); let outcome = checkpoint_session .checkpoint_table_with_wait(table_id) .await @@ -6525,7 +6572,7 @@ mod tests { drop(page); } assert_eq!( - engine.inner().mem_pool.allocated(), + engine.inner().pools.mem.allocated(), allocated_before_checkpoint, "checkpoint-retired pages must remain allocated while the reader pins system CTS eligibility" ); @@ -6536,7 +6583,7 @@ mod tests { .await .unwrap(); assert!( - engine.inner().mem_pool.allocated() < allocated_before_checkpoint, + engine.inner().pools.mem.allocated() < allocated_before_checkpoint, "checkpoint-retired pages must be deallocated after the reader releases the horizon" ); } @@ -7168,6 +7215,8 @@ mod tests { let mut old_session = engine.new_session().unwrap(); let old_trx = old_session.begin_trx().unwrap(); let retained_visible = engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, old_trx.sts()) .unwrap(); @@ -7180,6 +7229,8 @@ mod tests { let table = table_for_internal_assertion(&engine, table_id); let after_drop_root = table.file().active_root_unchecked().clone(); let CurrentTableState::Live { metadata, .. } = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -7195,7 +7246,11 @@ mod tests { "DROP INDEX detaches the root but leaves page reclamation to checkpoint reachability" ); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), Some(1) ); assert!(retained_live.metadata().idx.index_spec(0).is_some()); @@ -7218,7 +7273,11 @@ mod tests { } } assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), Some(0) ); assert!(retained_live.metadata().idx.index_spec(0).is_some()); diff --git a/doradb-storage/src/table/recover.rs b/doradb-storage/src/table/recover.rs index 40f40e94..dcfd3652 100644 --- a/doradb-storage/src/table/recover.rs +++ b/doradb-storage/src/table/recover.rs @@ -568,6 +568,8 @@ mod tests { let (table_spec, index_specs) = drop_table_test_spec(); let table_id = session.create_table(table_spec, index_specs).await.unwrap(); let table_for_internal_lifecycle = engine + .inner() + .core .catalog() .get_table_now(table_id) .expect("created table should still be loaded"); @@ -588,6 +590,8 @@ mod tests { .await .unwrap(); let current = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap(); @@ -598,10 +602,14 @@ mod tests { &table.metadata() )); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), Some(0) ); - assert_no_dropped_table_operational_state(engine.catalog(), table_id); + assert_no_dropped_table_operational_state(engine.inner().core.catalog(), table_id); }); } @@ -633,29 +641,56 @@ mod tests { )) .await .unwrap(); - assert!(engine.catalog().get_table(table_id).await.is_none()); - assert!(engine.catalog().get_table_now(table_id).is_none()); assert!( engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_none() + ); + assert!( + engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .is_none() + ); + assert!( + engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .is_none() ); assert!( engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, MAX_SNAPSHOT_TS) .is_none() ); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), None ); assert_eq!( - engine.catalog().retained_dropped_table_ids_now(), + engine + .inner() + .core + .catalog() + .retained_dropped_table_ids_now(), vec![table_id] ); - assert_dropped_table_floor(engine.catalog(), table_id); + assert_dropped_table_floor(engine.inner().core.catalog(), table_id); assert!(std::path::Path::new(&table_file_path).exists()); let mut session = engine.new_session().unwrap(); let (table_spec, index_specs) = drop_table_test_spec(); @@ -668,10 +703,21 @@ mod tests { .unwrap(); wait_for_no_dropped_table_operational_state(&engine, table_id).await; assert!(!std::path::Path::new(&table_file_path).exists()); - assert!(engine.catalog().retained_dropped_table_ids_now().is_empty()); - assert_no_dropped_table_operational_state(engine.catalog(), table_id); + assert!( + engine + .inner() + .core + .catalog() + .retained_dropped_table_ids_now() + .is_empty() + ); + assert_no_dropped_table_operational_state(engine.inner().core.catalog(), table_id); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), None ); }); @@ -711,7 +757,15 @@ mod tests { )) .await .unwrap(); - assert!(engine.catalog().get_table(table_id).await.is_none()); + assert!( + engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .is_none() + ); wait_path_exists(&table_file_path, false).await; }); } diff --git a/doradb-storage/src/trx/admission.rs b/doradb-storage/src/trx/admission.rs index ef6ebf22..b8f016c7 100644 --- a/doradb-storage/src/trx/admission.rs +++ b/doradb-storage/src/trx/admission.rs @@ -434,7 +434,7 @@ mod tests { resource: LockResource, mode: LockMode, ) -> bool { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .any(|entry| { @@ -461,7 +461,7 @@ mod tests { futures::poll!(future.as_mut()), std::task::Poll::Pending )); - if debug_snapshot(engine.lock_manager()) + if debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .any(|entry| { @@ -480,7 +480,7 @@ mod tests { let metadata = LockResource::TableMetadata(table_id); let data = LockResource::TableData(table_id); assert!( - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .all(|entry| entry.resource != metadata && entry.resource != data) @@ -519,7 +519,7 @@ mod tests { LockMode::Shared )); assert!( - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .all(|entry| { @@ -635,7 +635,7 @@ mod tests { LockMode::IntentExclusive )); } - let snapshot = debug_snapshot(engine.lock_manager()); + let snapshot = debug_snapshot(engine.inner().core.lock_manager()); assert!( snapshot .entries @@ -801,10 +801,17 @@ mod tests { .unwrap(); let mut ddl_session = engine.new_session().unwrap(); - let table = engine.catalog().get_table_now(table_id).unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .unwrap(); let before_layout = table.layout_snapshot(); let before_root = table.file().active_root_unchecked().clone(); let before_current_cts = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -816,6 +823,8 @@ mod tests { observe_metadata_x_waiter(&engine, metadata, create.as_mut()).await; assert_eq!( engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -855,7 +864,12 @@ mod tests { ) .await .unwrap(); - let table = engine.catalog().get_table_now(table_id).unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .unwrap(); let mut bound_session = engine.new_session().unwrap(); let bound_session_id = bound_session.id(); @@ -866,6 +880,8 @@ mod tests { .await .unwrap(); let before_current_cts = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -882,6 +898,8 @@ mod tests { LockMode::Shared )); let waiting_current = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap(); @@ -906,6 +924,8 @@ mod tests { bound_trx.commit().await.unwrap(); drop_index.await.unwrap(); let current = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap(); @@ -928,7 +948,12 @@ mod tests { let (_temp_dir, engine) = test_engine("admission_drop_table_waits_for_binding").await; let table_id = table2(&engine).await; let metadata_resource = LockResource::TableMetadata(table_id); - let table = engine.catalog().get_table_now(table_id).unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .unwrap(); let mut bound_session = engine.new_session().unwrap(); let bound_session_id = bound_session.id(); let mut bound_trx = bound_session.begin_trx().unwrap(); @@ -938,11 +963,17 @@ mod tests { .await .unwrap(); let before_current_cts = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() .effective_cts(); - let before_history_count = engine.catalog().user_table_history_version_count(table_id); + let before_history_count = engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id); let before_generation = table.layout_snapshot().generation(); let before_root_ts = table.file().active_root_unchecked().root_ts; @@ -956,6 +987,8 @@ mod tests { LockMode::Shared )); let waiting_current = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap(); @@ -966,7 +999,11 @@ mod tests { .is_some_and(|current| Arc::ptr_eq(current, &table)) ); assert_eq!( - engine.catalog().user_table_history_version_count(table_id), + engine + .inner() + .core + .catalog() + .user_table_history_version_count(table_id), before_history_count ); assert_eq!(table.layout_snapshot().generation(), before_generation); @@ -978,10 +1015,21 @@ mod tests { bound_trx.rollback().await.unwrap(); drop_table.await.unwrap(); assert!(!matches!( - engine.catalog().resolve_user_table_current(table_id), + engine + .inner() + .core + .catalog() + .resolve_user_table_current(table_id), Some(CurrentTableState::Live { .. }) )); - assert!(engine.catalog().get_table_now(table_id).is_none()); + assert!( + engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .is_none() + ); assert_no_table_locks(&engine, table_id); drop(ddl_session); @@ -1004,6 +1052,8 @@ mod tests { let mut ddl_session = engine.new_session().unwrap(); ddl_session.drop_index(table_id, 0).await.unwrap(); let visible = engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, old_sts) .unwrap(); @@ -1012,6 +1062,8 @@ mod tests { }; assert!(visible.metadata().idx.index_spec(0).is_some()); let CurrentTableState::Live { metadata, .. } = engine + .inner() + .core .catalog() .resolve_user_table_current(table_id) .unwrap() @@ -1045,7 +1097,7 @@ mod tests { ); } assert!( - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .all(|entry| entry.owner != old_owner @@ -1075,11 +1127,17 @@ mod tests { let mut ddl_session = engine.new_session().unwrap(); ddl_session.drop_table(table_id).await.unwrap(); assert!(matches!( - engine.catalog().resolve_user_table_current(table_id), + engine + .inner() + .core + .catalog() + .resolve_user_table_current(table_id), Some(CurrentTableState::Dropped { .. }) )); assert!(matches!( engine + .inner() + .core .catalog() .resolve_user_table_visible(table_id, old_sts), Some(ResolvedVisibleTableMetadata::Live(_)) @@ -1107,7 +1165,7 @@ mod tests { ); } assert!( - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .all(|entry| entry.owner != old_owner diff --git a/doradb-storage/src/trx/group.rs b/doradb-storage/src/trx/group.rs index 2d997090..fba42e49 100644 --- a/doradb-storage/src/trx/group.rs +++ b/doradb-storage/src/trx/group.rs @@ -267,7 +267,6 @@ mod tests { redo_bin: Some(redo_bin(cts)), payload: None, attachment: None, - lock_manager: None, lock_state: None, trx_inner: None, } @@ -279,7 +278,6 @@ mod tests { redo_bin: Some(redo_bin_large(cts)), payload: None, attachment: None, - lock_manager: None, lock_state: None, trx_inner: None, } @@ -291,7 +289,6 @@ mod tests { redo_bin: None, payload: None, attachment: None, - lock_manager: None, lock_state: None, trx_inner: None, } diff --git a/doradb-storage/src/trx/mod.rs b/doradb-storage/src/trx/mod.rs index ebb7fd6b..ffb1cd51 100644 --- a/doradb-storage/src/trx/mod.rs +++ b/doradb-storage/src/trx/mod.rs @@ -37,7 +37,7 @@ use crate::buffer::PoolGuards; use crate::buffer::page::VersionedPageID; use crate::catalog::{TableCache, is_catalog_table}; use crate::completion::Completion; -use crate::engine::{EngineRef, WeakEngineRef}; +use crate::engine::EngineCore; use crate::error::{ CompletionErrorBridge, DiscloseError, DiscloseResultExt, Error, FatalError, LifecycleError, LifecycleResult, OperationResult, ResourceError, Result, RuntimeError, RuntimeOrFatalError, @@ -45,16 +45,15 @@ use crate::error::{ }; use crate::id::{SessionID, SessionOperationKey, TableID, TrxID}; use crate::lock::{ - FreshLockGuard, LockManager, LockMode, LockOwner, LockResource, LockScope, OwnerLockState, - StmtNo, TableLockMode, + FreshLockGuard, LockMode, LockOwner, LockResource, LockScope, OwnerLockState, StmtNo, + TableLockMode, }; use crate::log::block_group::TrxLog; use crate::log::redo::{DDLRedo, RedoHeader, RedoLogs, RedoTrxKind}; use crate::map::FastHashMap; use crate::notify::EventNotifyOnDrop; use crate::obs; -use crate::quiescent::QuiescentGuard; -use crate::session::TrxAttachment; +use crate::session::{SessionRuntime, TrxAttachment, WeakSessionRef}; use crate::trx::undo::{IndexPurgeEntry, IndexUndoLogs, RowUndoHead, RowUndoLogs, UndoStatus}; use error_stack::{Report, ResultExt}; use event_listener::{Event, EventListener}; @@ -86,7 +85,7 @@ pub(crate) const MIN_ACTIVE_TRX_ID: TrxID = TrxID::new((1 << 63) + 1); /// Proof that one transaction's owner-local logical lock state was drained. /// /// The proof is deliberately single-use and is minted only by terminal -/// transaction cleanup after the retained lock-manager guard is also dropped. +/// transaction cleanup through the terminal attachment's engine lock manager. pub(crate) struct ReleasedTransactionLocks { trx_id: TrxID, } @@ -114,7 +113,7 @@ pub struct Transaction { trx_id: TrxID, sts: TrxID, operation_key: SessionOperationKey, - engine: WeakEngineRef, + session: WeakSessionRef, terminal_started: bool, } @@ -122,7 +121,7 @@ impl Transaction { /// Create a weak public transaction facade for a stable session entry. #[inline] pub(crate) fn new( - engine: WeakEngineRef, + session: WeakSessionRef, operation_key: SessionOperationKey, trx_id: TrxID, sts: TrxID, @@ -131,76 +130,87 @@ impl Transaction { trx_id, sts, operation_key, - engine, + session, terminal_started: false, } } - /// Resolve this handle and build an operation-local runtime attachment. + /// Check out the mutable core for one crate-internal operation under admission. #[inline] - fn resolve_active(&self) -> LifecycleResult<(Arc, TrxAttachment)> { - let engine = self.engine.upgrade().attach_with(|| { + pub(crate) fn checkout(&mut self) -> LifecycleResult { + let admitted = self.session.acquire_admission().attach_with(|| { format!( - "operation_key={}, trx_id={}, phase=upgrade_engine_runtime", + "operation_key={}, trx_id={}", self.operation_key, self.trx_id ) })?; - let admission = engine.acquire_admission().attach_with(|| { - format!( - "operation_key={}, trx_id={}", + let admitted = admitted.upgrade().ok_or_else(|| { + Report::new(LifecycleError::TransactionDiscarded).attach(format!( + "operation_key={}, trx_id={}, reason=session_missing", self.operation_key, self.trx_id - ) + )) })?; - let (entry, session) = engine - .session_registry - .resolve_operation(self.operation_key)?; - drop(admission); - let attachment = TrxAttachment::new(engine, session, self.operation_key, self.trx_id); - Ok((entry, attachment)) + admitted + .runtime() + .poisoner + .ensure_healthy() + .change_context(LifecycleError::RuntimeUnavailable) + .attach_with(|| { + format!( + "operation_key={}, trx_id={}, phase=check_engine_health", + self.operation_key, self.trx_id + ) + })?; + let entry = admitted + .runtime() + .state() + .resolve_operation(self.operation_key) + .ok_or_else(|| { + Report::new(LifecycleError::TransactionDiscarded).attach(format!( + "operation_key={}, trx_id={}, reason=operation_entry_missing", + self.operation_key, self.trx_id + )) + })?; + let runtime = admitted.into_runtime(); + let attachment = TrxAttachment::new(runtime, self.operation_key, self.trx_id); + let checkout = SessionOperationCheckout::new(entry, attachment)?; + Ok(checkout) } - /// Resolve this handle for terminal or cleanup paths. + /// Check out this handle's exact entry and attachment for terminal paths. #[inline] - fn resolve_terminal(&self) -> LifecycleResult<(Arc, TrxAttachment)> { - let engine = self.engine.upgrade_for_terminal().attach_with(|| { - format!( - "operation_key={}, trx_id={}, phase=upgrade_engine_runtime", + fn checkout_terminal(&self) -> LifecycleResult<(Arc, TrxAttachment)> { + let runtime = self.session.upgrade_for_terminal().ok_or_else(|| { + Report::new(LifecycleError::TransactionDiscarded).attach(format!( + "operation_key={}, trx_id={}, reason=session_missing", self.operation_key, self.trx_id - ) + )) })?; - self.resolve_with_engine(engine) - } - - #[inline] - fn resolve_with_engine( - &self, - engine: EngineRef, - ) -> LifecycleResult<(Arc, TrxAttachment)> { - let (entry, session) = engine - .session_registry - .resolve_operation(self.operation_key)?; - let attachment = TrxAttachment::new(engine, session, self.operation_key, self.trx_id); + let entry = runtime + .state() + .resolve_operation(self.operation_key) + .ok_or_else(|| { + Report::new(LifecycleError::TransactionDiscarded).attach(format!( + "operation_key={}, trx_id={}, reason=operation_entry_missing", + self.operation_key, self.trx_id + )) + })?; + let attachment = TrxAttachment::new(runtime, self.operation_key, self.trx_id); Ok((entry, attachment)) } - /// Check out the mutable core for one crate-internal operation. - #[inline] - pub(crate) fn checkout(&mut self) -> LifecycleResult { - let (entry, attachment) = self.resolve_active()?; - SessionOperationCheckout::new(entry, attachment) - } - /// Claim this transaction for an explicit terminal operation. #[inline] - pub(crate) fn claim_terminal(&self) -> LifecycleResult { - let (entry, attachment) = self.resolve_terminal()?; + fn claim_terminal(mut self) -> LifecycleResult { + self.terminal_started = true; + let (entry, attachment) = self.checkout_terminal()?; SessionOperationCompletionClaim::terminal(entry, attachment) } /// Best-effort check that the transaction can still reach its engine. #[inline] - pub(crate) fn engine(&self) -> Option { - self.engine.upgrade_for_cleanup() + pub(crate) fn engine(&self) -> Option { + self.session.upgrade_for_terminal() } /// Returns this transaction's current status timestamp. @@ -336,139 +346,81 @@ impl Transaction { /// Commit the transaction. #[inline] pub async fn commit(self) -> Result { - let mut trx = self; - trx.terminal_started = true; - let engine = trx - .engine - .upgrade_for_terminal() - .attach_with(|| { - format!( - "operation=commit_active_transaction, session_id={}, trx_id={}, phase=upgrade_engine_runtime", - trx.operation_key.session_id(), trx.trx_id - ) - }) - .disclose()?; - let claim = trx + let claim = self .claim_terminal() .attach("operation=commit_active_transaction") .disclose()?; - engine.trx_sys.commit_transaction(claim).await + let trx_sys = claim.engine().trx_sys.clone(); + trx_sys.commit_transaction(claim).await } /// Rollback the transaction. #[inline] pub async fn rollback(self) -> Result<()> { - let mut trx = self; - trx.terminal_started = true; - let engine = trx - .engine - .upgrade_for_terminal() - .attach_with(|| { - format!( - "operation=rollback_active_transaction, session_id={}, trx_id={}, phase=upgrade_engine_runtime", - trx.operation_key.session_id(), trx.trx_id - ) - }) - .disclose()?; - let claim = trx + let claim = self .claim_terminal() .attach("operation=rollback_active_transaction") .disclose()?; - engine.trx_sys.rollback_transaction(claim).await.disclose() + let trx_sys = claim.engine().trx_sys.clone(); + trx_sys.rollback_transaction(claim).await.disclose() } /// Commit a catalog DDL transaction without crossing the public error boundary. #[inline] pub(crate) async fn commit_catalog_ddl(self) -> RuntimeOrFatalResult { - let mut trx = self; - trx.terminal_started = true; - let engine = trx - .engine - .upgrade_for_terminal() - .change_context(RuntimeError::CatalogAccess) - .attach_with(|| { - format!( - "operation=commit_catalog_ddl, session_id={}, trx_id={}, phase=upgrade_engine_runtime", - trx.operation_key.session_id(), trx.trx_id - ) - }) - .map_err(RuntimeOrFatalError::from)?; - let claim = trx + let session_id = self.operation_key.session_id(); + let trx_id = self.trx_id; + let claim = self .claim_terminal() .change_context(RuntimeError::CatalogAccess) .attach_with(|| { format!( "operation=commit_catalog_ddl, session_id={}, trx_id={}", - trx.operation_key.session_id(), - trx.trx_id + session_id, trx_id ) }) .map_err(RuntimeOrFatalError::from)?; - engine.trx_sys.commit_catalog_transaction(claim).await + let trx_sys = claim.engine().trx_sys.clone(); + trx_sys.commit_catalog_transaction(claim).await } /// Roll back a catalog DDL transaction without crossing the public error boundary. #[inline] pub(crate) async fn rollback_catalog_ddl(self) -> RuntimeOrFatalResult<()> { - let mut trx = self; - trx.terminal_started = true; - let engine = trx - .engine - .upgrade_for_terminal() - .change_context(RuntimeError::CatalogAccess) - .attach_with(|| { - format!( - "operation=rollback_catalog_ddl, session_id={}, trx_id={}, phase=upgrade_engine_runtime", - trx.operation_key.session_id(), trx.trx_id - ) - }) - .map_err(RuntimeOrFatalError::from)?; - let claim = trx + let session_id = self.operation_key.session_id(); + let trx_id = self.trx_id; + let claim = self .claim_terminal() .change_context(RuntimeError::CatalogAccess) .attach_with(|| { format!( "operation=rollback_catalog_ddl, session_id={}, trx_id={}", - trx.operation_key.session_id(), - trx.trx_id + session_id, trx_id ) }) .map_err(RuntimeOrFatalError::from)?; - engine.trx_sys.rollback_catalog_transaction(claim).await + let trx_sys = claim.engine().trx_sys.clone(); + trx_sys.rollback_catalog_transaction(claim).await } /// Roll back an engine-owned table-maintenance transaction without crossing /// the public error boundary. #[inline] pub(crate) async fn rollback_table_maintenance(self) -> RuntimeOrFatalResult<()> { - let mut trx = self; - trx.terminal_started = true; - let engine = trx - .engine - .upgrade_for_terminal() - .change_context(RuntimeError::TableAccess) - .attach_with(|| { - format!( - "operation=rollback_table_maintenance, session_id={}, trx_id={}, phase=upgrade_engine_runtime", - trx.operation_key.session_id(), trx.trx_id - ) - }) - .map_err(RuntimeOrFatalError::from)?; - let claim = trx + let session_id = self.operation_key.session_id(); + let trx_id = self.trx_id; + let claim = self .claim_terminal() .change_context(RuntimeError::TableAccess) .attach_with(|| { format!( "operation=rollback_table_maintenance, session_id={}, trx_id={}", - trx.operation_key.session_id(), - trx.trx_id + session_id, trx_id ) }) .map_err(RuntimeOrFatalError::from)?; - engine - .trx_sys - .rollback_table_maintenance_transaction(claim) - .await + let trx_sys = claim.engine().trx_sys.clone(); + trx_sys.rollback_table_maintenance_transaction(claim).await } } @@ -478,16 +430,13 @@ impl Drop for Transaction { if self.terminal_started { return; } - if let Some(engine) = self.engine.upgrade_for_cleanup() { - let abandoned = engine - .session_registry + if let Some(runtime) = self.session.upgrade_for_terminal() { + let abandoned = runtime + .state() .abandon_trx_handle(self.operation_key, self.trx_id); if abandoned { - engine.trx_sys.request_abandoned_trx_cleanup( - engine.clone(), - self.operation_key, - self.trx_id, - ); + let trx_sys = runtime.trx_sys.clone(); + trx_sys.request_abandoned_trx_cleanup(runtime, self.operation_key, self.trx_id); } } } @@ -835,7 +784,7 @@ impl<'r> TrxRuntime<'r> { /// Returns the crate-private engine runtime handle. #[inline] - pub(crate) fn engine(&self) -> &'r EngineRef { + pub(crate) fn engine(&self) -> &'r EngineCore { self.attachment.engine() } @@ -2065,7 +2014,7 @@ impl SessionOperationCompletionClaim { /// Returns the engine retained by this terminal or cleanup claim. #[inline] - pub(crate) fn engine(&self) -> &EngineRef { + pub(crate) fn engine(&self) -> &EngineCore { self.attachment .as_ref() .expect("active completion claim retains terminal attachment") @@ -2085,13 +2034,14 @@ impl SessionOperationCompletionClaim { /// Abandoned transaction cleanup job. /// -/// The job carries component access through `EngineRef`; its mandatory internal -/// permit is the background-work shutdown proof. The stable session operation -/// remains visible until the task has either claimed and rolled back the -/// abandoned transaction or found that it is no longer claimable. +/// The job carries exact state and component access through `SessionRuntime` +/// until that runtime moves into the terminal attachment. Its mandatory +/// internal permit is the background-work shutdown proof. The stable session +/// operation remains visible until the task has either claimed and rolled back +/// the abandoned transaction or found that it is no longer claimable. pub(crate) struct SessionOperationCleanupJob { - /// Shared engine access retained until cleanup resolves this job. - pub(crate) engine: EngineRef, + /// Exact session runtime retained until cleanup resolves or claims this job. + pub(crate) runtime: Option, /// Exact stable operation that owns the abandoned transaction. pub(crate) operation_key: SessionOperationKey, /// Exact transaction id validated by the entry cleanup claim. @@ -2460,22 +2410,13 @@ impl TrxInner { } #[inline] - fn checked_engine(&self, attachment: &TrxAttachment) -> EngineRef { + fn checked_engine<'a>(&self, attachment: &'a TrxAttachment) -> &'a EngineCore { assert!( self.active, "checked-out transaction must retain its active core and engine attachment: trx_id={}", self.trx_id() ); - attachment.engine().clone() - } - - #[inline] - fn clone_lock_manager_guard( - &self, - attachment: &TrxAttachment, - ) -> Option> { - self.active - .then(|| attachment.engine().lock_manager().clone()) + attachment.engine() } #[inline] @@ -2680,7 +2621,6 @@ impl TrxInner { self.clear_table_bindings(); // fast path for readonly transactions if !self.require_ordered_commit() { - let lock_manager = self.clone_lock_manager_guard(&attachment); // there should be no ref count of transaction status. debug_assert!(Arc::strong_count(&self.ctx.status) == 1); debug_assert!(self.effects.index_undo.is_empty()); @@ -2697,7 +2637,6 @@ impl TrxInner { redo_bin: None, payload: Some(payload), attachment: Some(attachment), - lock_manager, lock_state, trx_inner: Some(self), }; @@ -2712,7 +2651,6 @@ impl TrxInner { None }; let (row_undo, index_undo) = self.effects.take_payload_parts(); - let lock_manager = self.clone_lock_manager_guard(&attachment); let payload = PreparedTrxPayload::User { status: Arc::clone(self.ctx.status()), sts: self.ctx.sts(), @@ -2726,7 +2664,6 @@ impl TrxInner { redo_bin, payload: Some(payload), attachment: Some(attachment), - lock_manager, lock_state, trx_inner: Some(self), } @@ -2784,7 +2721,6 @@ pub(crate) struct PreparedTrx { payload: Option, /// Terminal session attachment carried until ordered commit or rollback cleanup. pub(crate) attachment: Option, - lock_manager: Option>, lock_state: Option, /// Emptied session transaction core retained until the terminal outcome. trx_inner: Option>, @@ -2852,7 +2788,6 @@ impl PreparedTrx { redo_bin, payload, attachment: self.attachment.take(), - lock_manager: self.lock_manager.take(), lock_state: self.lock_state.take(), trx_inner: self.trx_inner.take(), } @@ -2861,7 +2796,7 @@ impl PreparedTrx { /// Releases and drops transaction-owned locks for an unordered discard path. #[inline] pub(self) fn release_transaction_locks(&mut self) -> Option { - release_carried_transaction_locks(&mut self.lock_state, &mut self.lock_manager) + release_carried_transaction_locks(self.attachment.as_ref(), &mut self.lock_state) } } @@ -2871,10 +2806,6 @@ impl Drop for PreparedTrx { assert!(self.redo_bin.is_none(), "redo should be cleared"); assert!(self.payload.is_none(), "payload should be cleared"); assert!(self.attachment.is_none(), "attachment should be cleared"); - assert!( - self.lock_manager.is_none(), - "lock manager should be cleared" - ); assert!(self.lock_state.is_none(), "lock state should be cleared"); assert!( self.trx_inner.is_none(), @@ -2925,12 +2856,12 @@ impl PrecommitTrxPayload { panic!("rollback requires a user precommit payload") }; let trx_sys = &attachment.engine().trx_sys; - let pool_guards = attachment.pool_guards().clone(); + let pool_guards = attachment.pool_guards(); let mut table_cache = TableCache::new(&trx_sys.catalog); index_undo - .rollback(&mut table_cache, &pool_guards, *sts) + .rollback(&mut table_cache, pool_guards, *sts) .await?; - row_undo.rollback(&mut table_cache, &pool_guards).await + row_undo.rollback(&mut table_cache, pool_guards).await } #[inline] @@ -2978,8 +2909,6 @@ pub(crate) struct PrecommitTrx { pub(crate) payload: Option, /// Terminal session attachment for user transactions. pub(crate) attachment: Option, - /// Lock manager retained to release transaction-owned locks. - pub(crate) lock_manager: Option>, /// Transaction-owned lock state retained until terminal cleanup. pub(crate) lock_state: Option, /// Emptied session transaction core retained until the terminal outcome. @@ -3095,10 +3024,6 @@ impl PrecommitTrx { #[inline] async fn rollback_failed_precommit(&mut self) -> FailedPrecommitRollbackOutcome { self.redo_bin.take(); - let engine = self - .attachment - .as_ref() - .map(|attachment| attachment.engine().clone()); if let (Some(payload), Some(attachment)) = (self.payload.as_mut(), self.attachment.as_ref()) { if let Err(err) = payload.rollback(attachment).await { @@ -3110,7 +3035,7 @@ impl PrecommitTrx { report ); let _ = attachment.engine().poisoner.poison(report); - self.finish_failed_precommit_with_retention(engine); + self.finish_failed_precommit_with_retention(); return FailedPrecommitRollbackOutcome::FailedRetained; } payload.record_rollback_for_purge(attachment); @@ -3127,21 +3052,18 @@ impl PrecommitTrx { #[inline] fn retain_failed_precommit_without_rollback(&mut self) { self.redo_bin.take(); - let engine = self - .attachment - .as_ref() - .map(|attachment| attachment.engine().clone()); - self.finish_failed_precommit_with_retention(engine); + self.finish_failed_precommit_with_retention(); } #[inline] - fn finish_failed_precommit_with_retention(&mut self, engine: Option) { + fn finish_failed_precommit_with_retention(&mut self) { let released = self.release_transaction_locks(); - self.finish_carried_session_rollback_without_reuse(released); + let attachment = self.finish_carried_session_rollback_without_reuse(released); if let Some(payload) = self.payload.take() { payload.release_prepare_waiters(); - if let Some(engine) = engine { - engine + if let Some(attachment) = attachment { + attachment + .engine() .trx_sys .retain_fatal_rollback(FatalRollbackRetention::Precommit(payload)); } else { @@ -3184,7 +3106,7 @@ impl PrecommitTrx { #[inline] fn release_transaction_locks(&mut self) -> Option { - release_carried_transaction_locks(&mut self.lock_state, &mut self.lock_manager) + release_carried_transaction_locks(self.attachment.as_ref(), &mut self.lock_state) } #[inline] @@ -3215,12 +3137,13 @@ impl PrecommitTrx { fn finish_carried_session_rollback_without_reuse( &mut self, released: Option, - ) { + ) -> Option { let inner = self.trx_inner.take(); match (self.attachment.take(), released) { (Some(attachment), Some(released)) => { attachment.rollback_without_reuse(released); drop(inner); + Some(attachment) } (None, None) => { assert!( @@ -3228,6 +3151,7 @@ impl PrecommitTrx { "system precommit cannot carry a session transaction core: cts={}", self.cts ); + None } (attachment, released) => { panic!( @@ -3249,10 +3173,6 @@ impl Drop for PrecommitTrx { assert!(self.redo_bin.is_none(), "redo should be cleared"); assert!(self.payload.is_none(), "payload should be cleared"); assert!(self.attachment.is_none(), "attachment should be cleared"); - assert!( - self.lock_manager.is_none(), - "lock manager should be cleared" - ); assert!(self.lock_state.is_none(), "lock state should be cleared"); assert!( self.trx_inner.is_none(), @@ -3368,30 +3288,29 @@ fn session_operation_entry_state_err( #[inline] fn release_carried_transaction_locks( + attachment: Option<&TrxAttachment>, lock_state: &mut Option, - lock_manager: &mut Option>, ) -> Option { - match (lock_state.take(), lock_manager.take()) { - (Some(mut lock_state), Some(lock_manager)) => { + match (lock_state.take(), attachment) { + (Some(mut lock_state), Some(attachment)) => { let owner = lock_state.owner(); let LockScope::Transaction(trx_id) = owner.scope() else { panic!("carried terminal lock state requires a transaction owner: owner={owner}") }; - lock_state.release_all(&lock_manager); + lock_state.release_all(attachment.engine().lock_manager()); lock_state.assert_cleared(); drop(lock_state); - drop(lock_manager); Some(ReleasedTransactionLocks::new(trx_id)) } (None, None) => None, (Some(lock_state), None) => { panic!( - "carried transaction lock state requires a lock-manager guard: owner={}", + "carried transaction lock state requires an attachment: owner={}", lock_state.owner() ) } (None, Some(_)) => { - panic!("carried transaction lock-manager guard requires owner lock state") + panic!("carried transaction attachment requires owner lock state") } } } @@ -3430,11 +3349,13 @@ pub(crate) mod tests { IOKind, StdIoResult, StorageBackendFileIdentity, StorageBackendOp, StorageBackendTestHook, install_storage_backend_test_hook, }; + use crate::lock::LockManager; use crate::lock::tests::{LockDebugEntryState, debug_snapshot, try_acquire}; use crate::log::redo::{RowRedo, RowRedoKind}; + use crate::quiescent::QuiescentGuard; use crate::row::ops::SelectKey; use crate::session::{ - Session, + Session, SessionRegistry, SessionShutdownWait, tests::{ SessionTestExt, TerminalAttachmentOutcome, TerminalAttachmentTestHookGuard, active_operation_count, assert_existing_transaction_error, @@ -3747,14 +3668,19 @@ pub(crate) mod tests { fn resolve_active_parts_for_test( trx: &Transaction, ) -> Result<(Arc, TrxAttachment)> { - let engine = trx - .engine + let runtime = trx + .session .upgrade_for_terminal() - .attach_with(|| format!("operation_key={}, trx_id={}", trx.operation_key, trx.trx_id)) + .ok_or_else(|| { + Report::new(LifecycleError::TransactionDiscarded).attach(format!( + "operation_key={}, trx_id={}, reason=session_missing", + trx.operation_key, trx.trx_id + )) + }) .disclose()?; - let (entry, session) = engine - .session_registry - .try_resolve_operation(trx.operation_key) + let entry = runtime + .state() + .resolve_operation(trx.operation_key) .ok_or_else(|| { Report::new(LifecycleError::TransactionDiscarded).attach(format!( "operation_key={}, trx_id={}, reason=transaction_not_resolvable", @@ -3762,7 +3688,7 @@ pub(crate) mod tests { )) }) .disclose()?; - let attachment = TrxAttachment::new(engine, session, trx.operation_key, trx.trx_id); + let attachment = TrxAttachment::new(runtime, trx.operation_key, trx.trx_id); Ok((entry, attachment)) } @@ -4129,15 +4055,12 @@ pub(crate) mod tests { test_engine("same_operation_key_rejects_wrong_transaction_identity").await; let mut session = engine.new_session().unwrap(); let trx = session.begin_trx().unwrap(); - let engine_ref = engine.new_ref().unwrap(); let mut forged = Transaction::new( - engine_ref.downgrade(), + trx.session.clone(), trx.operation_key, TrxID::new(trx.trx_id().as_u64() + 1), trx.sts(), ); - drop(engine_ref); - let err = match forged.checkout() { Ok(_) => panic!("wrong transaction id must not claim the exact operation entry"), Err(err) => err, @@ -4180,12 +4103,13 @@ pub(crate) mod tests { .session_registry .first_shutdown_wait() .expect("checked-out transaction must block shutdown"); - assert_eq!(shutdown_wait.blocker.cleanup(), None); + let SessionShutdownWait { blocker, listener } = shutdown_wait; + assert!(blocker.into_cleanup().is_none()); let (ready_tx, ready_rx) = mpsc::channel(); let (done_tx, done_rx) = mpsc::channel(); let waiter = spawn(move || { ready_tx.send(()).expect("waiter should report ready"); - shutdown_wait.listener.wait(); + listener.wait(); done_tx.send(()).expect("waiter should report completion"); }); @@ -4282,8 +4206,7 @@ pub(crate) mod tests { } #[inline] - fn prepare_transaction(mut trx: Transaction) -> Result { - trx.terminal_started = true; + fn prepare_transaction(trx: Transaction) -> Result { let claim = trx .claim_terminal() .attach("operation=prepare_active_transaction") @@ -4350,7 +4273,9 @@ pub(crate) mod tests { let engine = trx.engine().expect("test transaction must have engine"); discard_transaction_after_fatal_rollback(trx); engine.trx_sys.record_rollback_for_purge(gc_no, sts); - remove_session_for_test(&engine.session_registry, session_id); + if let Some(registry) = engine.session_registry.upgrade() { + remove_session_for_test(®istry, session_id); + } } /// Add one redo log entry for tests that need a non-readonly transaction. @@ -4451,7 +4376,7 @@ pub(crate) mod tests { } fn lock_entry_count(engine: &Engine, owner: LockOwner) -> usize { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .filter(|entry| entry.owner == owner) @@ -4468,7 +4393,8 @@ pub(crate) mod tests { } fn install_terminal_boundary_observer( - engine: EngineRef, + lock_manager: QuiescentGuard, + session_registry: Arc, operation_key: SessionOperationKey, target_trx_id: TrxID, status: Option>, @@ -4483,7 +4409,7 @@ pub(crate) mod tests { if trx_id != target_trx_id { return; } - let snapshot = debug_snapshot(engine.lock_manager()); + let snapshot = debug_snapshot(&lock_manager); let transaction_lock_entries = snapshot .entries .iter() @@ -4500,8 +4426,7 @@ pub(crate) mod tests { .send(TerminalBoundaryObservation { outcome, transaction_lock_entries, - session_active: engine - .session_registry + session_active: session_registry .try_resolve_operation(operation_key) .is_some(), status_ts: status.as_ref().map(|status| status.ts()), @@ -4586,7 +4511,7 @@ pub(crate) mod tests { mode: LockMode, state: LockDebugEntryState, ) -> bool { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .any(|entry| { @@ -4598,7 +4523,7 @@ pub(crate) mod tests { } fn has_lock_resource(engine: &Engine, owner: LockOwner, resource: LockResource) -> bool { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .any(|entry| entry.owner == owner && entry.resource == resource) @@ -5209,7 +5134,7 @@ pub(crate) mod tests { let blocker = LockOwner::transaction(SessionID::new(id), TrxID::new(id)); assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), resource, LockMode::Exclusive, blocker @@ -5233,7 +5158,14 @@ pub(crate) mod tests { LockDebugEntryState::Waiting, )); if promote_before_drop { - assert_eq!(engine.lock_manager().release(resource, blocker), 1); + assert_eq!( + engine + .inner() + .core + .lock_manager() + .release(resource, blocker), + 1 + ); assert!(has_lock_entry( &engine, stmt_owner, @@ -5247,7 +5179,14 @@ pub(crate) mod tests { assert!(!has_lock_resource(&engine, stmt_owner, resource)); if !promote_before_drop { - assert_eq!(engine.lock_manager().release(resource, blocker), 1); + assert_eq!( + engine + .inner() + .core + .lock_manager() + .release(resource, blocker), + 1 + ); } let err = trx.rollback().await.unwrap_err(); assert_eq!( @@ -5459,7 +5398,7 @@ pub(crate) mod tests { assert!(try_acquire_transaction_lock(&mut trx, metadata, LockMode::Shared).unwrap()); assert!( try_acquire( - engine.lock_manager(), + engine.inner().core.lock_manager(), metadata, LockMode::Shared, LockOwner::session_explicit(SessionID::new(91_221)) @@ -5473,6 +5412,8 @@ pub(crate) mod tests { Some(OperationError::LockUpgradeWouldBlock) ); engine + .inner() + .core .lock_manager() .release_owner(LockOwner::session_explicit(SessionID::new(91_221))); @@ -5548,7 +5489,13 @@ pub(crate) mod tests { let blocker = LockOwner::transaction(SessionID::new(91_401), TrxID::new(91_401)); let data = LockResource::TableData(table_id); assert!( - try_acquire(engine.lock_manager(), data, LockMode::Exclusive, blocker).unwrap() + try_acquire( + engine.inner().core.lock_manager(), + data, + LockMode::Exclusive, + blocker + ) + .unwrap() ); let mut session = engine.new_session().unwrap(); @@ -5590,7 +5537,7 @@ pub(crate) mod tests { assert!(!has_lock_resource(&engine, owner, metadata)); assert!(!has_lock_resource(&engine, owner, data)); assert!(!cached_transaction_lock_covers(&trx, metadata, LockMode::Shared).unwrap()); - assert_eq!(engine.lock_manager().release(data, blocker), 1); + assert_eq!(engine.inner().core.lock_manager().release(data, blocker), 1); trx.rollback().await.unwrap(); }); @@ -5613,7 +5560,13 @@ pub(crate) mod tests { let blocker = LockOwner::transaction(SessionID::new(91_402), TrxID::new(91_402)); assert!( - try_acquire(engine.lock_manager(), data, LockMode::Exclusive, blocker).unwrap() + try_acquire( + engine.inner().core.lock_manager(), + data, + LockMode::Exclusive, + blocker + ) + .unwrap() ); let mut lock_fut = Box::pin(trx.lock_table(table_id, TableLockMode::Shared)); @@ -5656,7 +5609,7 @@ pub(crate) mod tests { assert!(!has_lock_resource(&engine, owner, data)); assert!(cached_transaction_lock_covers(&trx, metadata, LockMode::Shared).unwrap()); assert!(!cached_transaction_lock_covers(&trx, data, LockMode::Shared).unwrap()); - assert_eq!(engine.lock_manager().release(data, blocker), 1); + assert_eq!(engine.inner().core.lock_manager().release(data, blocker), 1); trx.rollback().await.unwrap(); assert_eq!(lock_entry_count(&engine, owner), 0); @@ -5752,7 +5705,8 @@ pub(crate) mod tests { }) .unwrap(); let (hook, observed_rx) = install_terminal_boundary_observer( - engine.new_ref().unwrap(), + engine.inner().core.lock_manager().clone(), + Arc::clone(&engine.inner().session_registry), trx.operation_key, trx_id, Some(status), @@ -5788,7 +5742,8 @@ pub(crate) mod tests { .unwrap() ); let (hook, observed_rx) = install_terminal_boundary_observer( - engine.new_ref().unwrap(), + engine.inner().core.lock_manager().clone(), + Arc::clone(&engine.inner().session_registry), trx.operation_key, trx_id, None, @@ -5823,7 +5778,8 @@ pub(crate) mod tests { .unwrap() ); let (hook, observed_rx) = install_terminal_boundary_observer( - engine.new_ref().unwrap(), + engine.inner().core.lock_manager().clone(), + Arc::clone(&engine.inner().session_registry), trx.operation_key, trx_id, None, @@ -5928,7 +5884,8 @@ pub(crate) mod tests { .unwrap() ); let (hook, observed_rx) = install_terminal_boundary_observer( - engine.new_ref().unwrap(), + engine.inner().core.lock_manager().clone(), + Arc::clone(&engine.inner().session_registry), trx.operation_key, trx_id, None, @@ -5972,7 +5929,8 @@ pub(crate) mod tests { let prepared = prepare_transaction(trx).unwrap(); let mut precommit = prepared.fill_cts(TrxID::new(91_241)); let (hook, observed_rx) = install_terminal_boundary_observer( - engine.new_ref().unwrap(), + engine.inner().core.lock_manager().clone(), + Arc::clone(&engine.inner().session_registry), operation_key, trx_id, None, @@ -6135,13 +6093,12 @@ pub(crate) mod tests { .expect("terminal rollback worker should start"); assert_eq!(entry.inspect().state, SessionOperationState::Completing); - let engine_ref = engine.new_ref().unwrap(); - let (duplicate_entry, duplicate_session) = engine_ref - .session_registry - .try_resolve_operation(operation_key) - .expect("rolling-back transaction should remain registry-visible"); - let duplicate_attachment = - TrxAttachment::new(engine_ref, duplicate_session, operation_key, trx_id); + let duplicate_runtime = session.engine(); + let duplicate_entry = duplicate_runtime + .state() + .resolve_operation(operation_key) + .expect("rolling-back transaction should retain its exact entry"); + let duplicate_attachment = TrxAttachment::new(duplicate_runtime, operation_key, trx_id); assert!( SessionOperationCompletionClaim::cleanup(duplicate_entry, duplicate_attachment) .is_err(), @@ -6229,7 +6186,13 @@ pub(crate) mod tests { ) .await; let table_id = catalog_tests::table2(&engine).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let large = "r".repeat(48 * 1024); fn precommit_with_cold_row_undo( @@ -6504,7 +6467,7 @@ pub(crate) mod tests { let mutable = MutableTableFile::fork( &table_file, engine.inner().table_fs.background_writes(), - engine.inner().disk_pool.clone_inner(), + engine.inner().pools.disk.clone(), ); let table_file = engine .inner() diff --git a/doradb-storage/src/trx/purge.rs b/doradb-storage/src/trx/purge.rs index d3c07051..f1db0555 100644 --- a/doradb-storage/src/trx/purge.rs +++ b/doradb-storage/src/trx/purge.rs @@ -1451,10 +1451,10 @@ mod tests { #[inline] fn full_pool_guards(engine: &Engine) -> PoolGuards { PoolGuards::builder() - .push(PoolRole::Meta, engine.inner().meta_pool.pool_guard()) - .push(PoolRole::Index, engine.inner().index_pool.pool_guard()) - .push(PoolRole::Mem, engine.inner().mem_pool.pool_guard()) - .push(PoolRole::Disk, engine.inner().disk_pool.pool_guard()) + .push(PoolRole::Meta, engine.inner().pools.meta.pool_guard()) + .push(PoolRole::Index, engine.inner().pools.index.pool_guard()) + .push(PoolRole::Mem, engine.inner().pools.mem.pool_guard()) + .push(PoolRole::Disk, engine.inner().pools.disk.pool_guard()) .build() } @@ -1646,7 +1646,12 @@ mod tests { let (_temp_dir, engine) = purge_test_engine("drop_runtime_unique_assertion", 1, 1).await; let table_id = table1(&engine).await; - let table = engine.catalog().get_table_now(table_id).unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table_now(table_id) + .unwrap(); let expected_strong_count = Arc::strong_count(&table) + 1; let panic = match catch_unwind(AssertUnwindSafe(|| { @@ -2319,7 +2324,7 @@ mod tests { engine .inner() .trx_sys - .process_retired_row_pages(engine.catalog(), &guards, Vec::new()) + .process_retired_row_pages(engine.inner().core.catalog(), &guards, Vec::new()) .await ); assert!(engine.inner().poisoner.poison_error().is_none()); @@ -2329,7 +2334,7 @@ mod tests { .inner() .trx_sys .process_retired_row_pages( - engine.catalog(), + engine.inner().core.catalog(), &guards, vec![RetiredRowPageBatch::new( TableID::new(999_999), @@ -2396,7 +2401,7 @@ mod tests { let err = engine .inner() .trx_sys - .purge_gc_bucket(engine.catalog(), &guards, 0, MAX_SNAPSHOT_TS) + .purge_gc_bucket(engine.inner().core.catalog(), &guards, 0, MAX_SNAPSHOT_TS) .await .unwrap_err(); assert_eq!( @@ -2442,7 +2447,13 @@ mod tests { .unwrap(); let table_id = table1(&engine).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); let mut trx = session.begin_trx().unwrap(); trx.exec(async |stmt| { @@ -2489,7 +2500,12 @@ mod tests { engine .inner() .trx_sys - .purge_trx_list(engine.catalog(), &pool_guards, vec![trx], MAX_SNAPSHOT_TS) + .purge_trx_list( + engine.inner().core.catalog(), + &pool_guards, + vec![trx], + MAX_SNAPSHOT_TS, + ) .await .unwrap(); } @@ -2526,7 +2542,13 @@ mod tests { .unwrap(); let table_id = table1(&engine).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); let mut trx = session.begin_trx().unwrap(); trx.exec(async |stmt| { @@ -2573,7 +2595,12 @@ mod tests { engine .inner() .trx_sys - .purge_trx_list(engine.catalog(), &pool_guards, vec![trx], MAX_SNAPSHOT_TS) + .purge_trx_list( + engine.inner().core.catalog(), + &pool_guards, + vec![trx], + MAX_SNAPSHOT_TS, + ) .await .unwrap(); } @@ -2614,7 +2641,13 @@ mod tests { .unwrap(); let table_id = table1(&engine).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); let mut trx = session.begin_trx().unwrap(); trx.exec(async |stmt| { @@ -2684,7 +2717,12 @@ mod tests { engine .inner() .trx_sys - .purge_trx_list(engine.catalog(), &pool_guards, vec![trx], MAX_SNAPSHOT_TS) + .purge_trx_list( + engine.inner().core.catalog(), + &pool_guards, + vec![trx], + MAX_SNAPSHOT_TS, + ) .await .unwrap(); } @@ -2721,7 +2759,13 @@ mod tests { .unwrap(); let table_id = table1(&engine).await; - let table = engine.catalog().get_table(table_id).await.unwrap(); + let table = engine + .inner() + .core + .catalog() + .get_table(table_id) + .await + .unwrap(); let mut session = engine.new_session().unwrap(); let mut trx = session.begin_trx().unwrap(); trx.exec(async |stmt| { @@ -2791,7 +2835,12 @@ mod tests { engine .inner() .trx_sys - .purge_trx_list(engine.catalog(), &pool_guards, vec![trx], MAX_SNAPSHOT_TS) + .purge_trx_list( + engine.inner().core.catalog(), + &pool_guards, + vec![trx], + MAX_SNAPSHOT_TS, + ) .await .unwrap(); } diff --git a/doradb-storage/src/trx/retention.rs b/doradb-storage/src/trx/retention.rs index 366dbb18..76ecf5d1 100644 --- a/doradb-storage/src/trx/retention.rs +++ b/doradb-storage/src/trx/retention.rs @@ -136,7 +136,7 @@ impl MaintenanceExecutionSpec for RedoTruncationExecution { resources: &mut Self::Resources, _panic_label: &mut Self::PanicLabel, ) -> CompletionResult { - let engine = scope.engine().clone(); + let engine = scope.engine(); let result = engine .trx_sys .truncate_redo_log_prepared( @@ -170,7 +170,7 @@ impl MaintenanceExecutionSpec for CatalogRedoMaintenanceExecution { resources: &mut Self::Resources, _panic_label: &mut Self::PanicLabel, ) -> CompletionResult { - let engine = scope.engine().clone(); + let engine = scope.engine(); let result = engine .trx_sys .checkpoint_catalog_and_truncate_redo_log_prepared( diff --git a/doradb-storage/src/trx/stmt.rs b/doradb-storage/src/trx/stmt.rs index cd5dfee1..00dfe2bb 100644 --- a/doradb-storage/src/trx/stmt.rs +++ b/doradb-storage/src/trx/stmt.rs @@ -986,7 +986,7 @@ impl<'stmt> Statement<'stmt> { #[inline] pub(crate) async fn rollback_effects(&mut self) -> FatalResult<()> { let sts = self.inner.sts(); - let engine = self.attachment.engine().clone(); + let engine = self.attachment.engine(); let pool_guards = self.attachment.pool_guards(); let mut table_cache = TableCache::new(engine.catalog()); if let Err(err) = self @@ -1183,11 +1183,9 @@ pub(crate) mod tests { } fn test_trx(engine: &Engine, sts: TrxID) -> (Transaction, Arc) { - let engine_ref = engine.new_ref().unwrap(); - let session_id = engine_ref.next_session_id(); + let session_id = engine.inner().next_session_id(); session_tests::create_test_transaction( - &engine.inner().session_registry, - engine_ref, + engine, session_id, MIN_ACTIVE_TRX_ID + sts.as_u64(), sts, @@ -1196,7 +1194,7 @@ pub(crate) mod tests { } fn lock_entry_count(engine: &Engine, owner: LockOwner) -> usize { - debug_snapshot(engine.lock_manager()) + debug_snapshot(engine.inner().core.lock_manager()) .entries .iter() .filter(|entry| entry.owner == owner) @@ -1266,7 +1264,7 @@ pub(crate) mod tests { .expect("test transaction should be available for checkout"); let sts = checkout.inner().sts(); let pool_guards = checkout.attachment().pool_guards().clone(); - let mut table_cache = TableCache::new(engine.catalog()); + let mut table_cache = TableCache::new(engine.inner().core.catalog()); let table_id = TableID::new(99_999_998); let row_id = RowID::new(23); let mut effects = StmtEffects::empty(); @@ -1330,6 +1328,8 @@ pub(crate) mod tests { smol::block_on(async { let (_temp_dir, engine) = test_engine("redo_catalog_delete_pk_mismatch").await; let catalog_table = engine + .inner() + .core .catalog() .storage .get_catalog_table(TABLE_ID_TABLES) diff --git a/doradb-storage/src/trx/sys.rs b/doradb-storage/src/trx/sys.rs index b038b345..3ca8feb0 100644 --- a/doradb-storage/src/trx/sys.rs +++ b/doradb-storage/src/trx/sys.rs @@ -6,7 +6,6 @@ use crate::component::{ Component, ComponentRegistry, EnginePools, IndexPool, MemPool, MetaPool, ShelfScope, Supplier, }; use crate::conf::TrxSysConfig; -use crate::engine::EngineRef; use crate::error::{ CompletionErrorBridge, DataIntegrityError, DataIntegrityResult, DiscloseError, DiscloseResultExt, Error, FatalError, FatalResult, MultiDomainResultExt, Result, RuntimeError, @@ -25,7 +24,7 @@ use crate::quiescent::{QuiescentBox, QuiescentGuard, SyncQuiescentGuard}; use crate::recovery::RecoveryResources; use crate::recovery::stream::CatalogSafeRedoSegment; use crate::runtime::mandatory::{MandatoryInternalTask, MandatoryRuntime, MandatoryTaskMetadata}; -use crate::session::TrxAttachment; +use crate::session::{SessionRuntime, TrxAttachment, WeakSessionRef}; use crate::thread; use crate::trx::group::{Commit, CommitJoin, GroupCommit}; #[cfg(test)] @@ -617,7 +616,7 @@ impl TransactionSystem { debug_assert!(config.recovery_io_depth != 0); debug_assert!(config.catalog_checkpoint_scan_io_depth != 0); - let pool_guards = pools.pool_guards(); + let pool_guards = pools.pool_guards().clone(); let (purge_tx, purge_rx) = flume::unbounded(); let file_prefix = config.file_prefix().disclose()?; let recovery_resources = RecoveryResources::new(pools, table_fs.clone(), &catalog); @@ -1069,13 +1068,13 @@ impl TransactionSystem { #[inline] pub(crate) fn begin_public_trx( &self, - engine: &EngineRef, + session: WeakSessionRef, operation_key: SessionOperationKey, mut inner: Box, ) -> (Transaction, Arc) { let (trx_id, sts) = self.init_trx(operation_key.session_id(), inner.as_mut()); let entry = SessionOperationEntry::new_public_transaction(operation_key, inner); - let handle = Transaction::new(engine.downgrade(), operation_key, trx_id, sts); + let handle = Transaction::new(session, operation_key, trx_id, sts); (handle, entry) } @@ -1083,14 +1082,14 @@ impl TransactionSystem { #[inline] pub(crate) fn begin_private_trx( &self, - engine: &EngineRef, + session: WeakSessionRef, enclosing_entry: &Arc, mut inner: Box, ) -> Transaction { let operation_key = enclosing_entry.key(); let (trx_id, sts) = self.init_trx(operation_key.session_id(), inner.as_mut()); enclosing_entry.install_private_transaction(inner); - Transaction::new(engine.downgrade(), operation_key, trx_id, sts) + Transaction::new(session, operation_key, trx_id, sts) } /// Allocate a timestamp fence for a runtime state transition. @@ -1346,11 +1345,11 @@ impl TransactionSystem { let sts = inner.sts(); let gc_no = inner.gc_no(); let status = Arc::clone(inner.ctx().status()); - let pool_guards = attachment.pool_guards().clone(); + let pool_guards = attachment.pool_guards(); let mut table_cache = TableCache::new(&self.catalog); if let Err(err) = inner .index_undo_mut() - .rollback(&mut table_cache, &pool_guards, sts) + .rollback(&mut table_cache, pool_guards, sts) .await { drop(table_cache); @@ -1372,7 +1371,7 @@ impl TransactionSystem { } if let Err(err) = inner .row_undo_mut() - .rollback(&mut table_cache, &pool_guards) + .rollback(&mut table_cache, pool_guards) .await { drop(table_cache); @@ -1592,14 +1591,14 @@ impl TransactionSystem { #[inline] pub(crate) fn request_abandoned_trx_cleanup( &self, - engine: EngineRef, + runtime: SessionRuntime, operation_key: SessionOperationKey, trx_id: TrxID, ) { let _ = self .mandatory_runtime .submit_internal(SessionOperationCleanupJob { - engine, + runtime: Some(runtime), operation_key, trx_id, claim: None, @@ -1736,24 +1735,33 @@ fn recovery_initial_trx_ts(max_recovered_cts: TrxID) -> DataIntegrityResult parts, + let entry = match runtime.state().resolve_operation(operation_key) { + Some(entry) => entry, None => return, }; - let attachment = TrxAttachment::new(engine.clone(), session, operation_key, trx_id); + let attachment = TrxAttachment::new(runtime, operation_key, trx_id); let claim = match SessionOperationCompletionClaim::cleanup(entry, attachment) { Ok(claim) => claim, Err(_) => return, }; job.claim = Some(claim); } - let trx_sys = engine.trx_sys.clone(); + let trx_sys = job + .claim + .as_ref() + .expect("claimed abandoned cleanup retains terminal ownership") + .engine() + .trx_sys + .clone(); let result = trx_sys .cleanup_abandoned_transaction( job.claim @@ -1992,10 +2000,10 @@ pub(crate) mod tests { fn capture_transaction_cleanup_state( trx: &Transaction, ) -> (Arc, Arc) { - let engine = trx.engine().expect("test transaction must have engine"); - let (entry, _session) = engine - .session_registry - .try_resolve_operation(trx.operation_key) + let runtime = trx.engine().expect("test transaction must have runtime"); + let entry = runtime + .state() + .resolve_operation(trx.operation_key) .expect("test transaction must resolve"); let status = { let inner_slot = entry.inner.lock(); @@ -2090,9 +2098,8 @@ pub(crate) mod tests { let _ = smol::block_on(trx.commit()); } { - let engine = engine.new_ref().unwrap(); + let mut session = engine.new_session().unwrap(); spawn(move || { - let mut session = engine.new_session().unwrap(); let trx = session.begin_trx().unwrap(); let _ = smol::block_on(trx.commit()); }) diff --git a/doradb-storage/src/trx/sys_trx.rs b/doradb-storage/src/trx/sys_trx.rs index 68f2e64a..7ee7855f 100644 --- a/doradb-storage/src/trx/sys_trx.rs +++ b/doradb-storage/src/trx/sys_trx.rs @@ -183,7 +183,6 @@ impl SysTrx { PreparedTrxPayload::System(SysTrxPayload { retired_row_pages }) }), attachment: None, - lock_manager: None, lock_state: None, trx_inner: None, }