Repository navigation
server-zd: default the log deletion delay to 1 hour in single-node modes - #1353
Merged
Merged
Conversation
Single-node modes have no standby and keep the embedded object store on the same disk as RocksDB. A 24-hour delay kept superseded log batches and snapshots for a day, so disk use grew to about 2-3 times a day's writes. serve keeps 24 hours, and an explicit value still wins in every mode.
…e disk use includes Log objects wait for the covering snapshot to reach the delay, counted from its creation time, and displaced snapshots wait for the newer one. The disk estimate now names the retained snapshots in the embedded object store.
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
dex-server-zd devanddex-server-zd localnow defaultsnapshot.logDeletionDelayto 1 hour.servekeeps the 24-hour default. An explicitly configured value still wins in every mode, and validation is unchanged (must be greater than zero). The single-node default follows the existing pattern inConfig::load_single_node(applied only when the overrides YAML does not set the key), like the lease duration and partition count defaults. The configuration reference gains a "Disk use" section for single-node mode and a comparison-table row.Why
A single-machine disk soak test showed that with the 24-hour default, the embedded s3local object store, which shares the disk with RocksDB, grew without bound for a full day. Superseded log batches and old snapshots stayed for 24 hours, about 2–3 times the day's write volume. With a short delay, disk use levelled off.
The delay protects three things:
Single-machine modes have no hot standby, so the second reason does not apply there. One hour still covers the first and third. Clustered
servedeployments keep 24 hours, where the extra object storage is cheap and a lagging standby benefits from log catch-up.Compatibility
This changes a default for
devandlocalonly. Configurations that setsnapshot.logDeletionDelayexplicitly are unaffected in every mode, andservedefaults are unchanged. No persistent format or API changes.Testing
single_node_log_deletion_delay_defaults_to_one_hour_unless_the_overrides_set_it:devandlocaldefault to 1 h, including when anothersnapshotkey is set; an explicit24hoverrides in both;servestays 24 h.cargo test -p dexzd-server --lib config::,cargo fmt --all --check,cargo clippy -p dexzd-server --all-targets -- -D warnings,make rust-rules-check,make copyright-check,cd docs && npm run check: all pass locally.