feat(node): add a durable table of blobs pending recovery - #3667
feat(node): add a durable table of blobs pending recovery#3667halfprice wants to merge 2 commits into
Conversation
Add a pending_recover_blobs column family that records blobs whose recovery is pending, keyed by blob ID with the certify event index and epoch. Includes the storage API, database options, config knobs for the upcoming drain executor, metrics, and a dbtool read command. The table is not yet populated; the write path and the executor that drains it land in a follow-up change.
|
Warning: This PR modifies one of the example config files. Please consider the
|
|
@codex review |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f64bd30a65
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ds on open A RocksDB read error is re-yielded by the iterator without advancing, so counting with `Iterator::count` would loop forever and hang the storage open. Propagate the first error instead.
|
@codex review |
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
1 participant
Description
First PR of a two-PR stack that decouples the persisted event cursor from blob recovery (design: record recovering blobs durably and always mark certify events as persisted).
This PR adds the inert plumbing only — no behavior change:
pending_recover_blobscolumn family andPendingRecoverBlobsTable(key: blob ID; value: versioned record with the certify event index and epoch), with a cached record count.allow(dead_code), removed by the follow-up PR that adds the write path and executor).blob_recoveryconfig knobs for the upcoming drain executor (max_concurrent_pending_recoveries,pending_recovery_drain_interval_secs).read-pending-recover-blobsdbtool command.Existing databases pick up the new column family automatically on restart; no migration is needed.
Test plan
walrus-servicesuite, clippy, and the config example snapshot test.