rptest: add test_toggle_reconciliation#31059
Merged
WillemKauf merged 1 commit intoJul 13, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a new end-to-end rptest that stress-tests cloud-topics reconciliation/retention by repeatedly disabling/enabling the reconciliation loop while producing under aggressive size-based retention, then validating that the remaining log is consumable.
Changes:
- Extend topic setup to pass through
retention.byteswhen configured onTopicSpec. - Enhance
wait_until_reconciledto optionally wait for L1 reconciliation to reach a specific target offset (not just tail). - Add
EndToEndCloudTopicsReconciliationToggleTest.test_toggle_reconciliation(parameterized over cloud + tiered cloud modes) to exercise rapid reconciliation toggling with retention enforcement.
Comment on lines
+165
to
+168
| # Check the last observable record's offset against the next offset | ||
| # expected. For transactions, this could be much less than the HWM | ||
| # if there are aborts. | ||
| return (get_next_offset() - 1) == get_last_record() |
Comment on lines
+1135
to
+1137
| # Rate-limit the produce so it spans the whole toggle window: at | ||
| # 1 MiB/s this is ~160s of traffic, vs. a 75s toggle window. | ||
| self.msg_count = 40_000 |
Adds a parameterized test for cloud topics and TSv2 which rapidly toggles the reconciler on and off while waiting for retention to be applied. Asserts that after rounds of producing, waiting for retention to apply, and toggling the reconciler, that all data eventually makes it to L1 and it is consumable.
WillemKauf
force-pushed
the
cloud_Topics_toggle_reconciler
branch
from
7721208 to
97efb1b
Compare
wdberkeley
approved these changes
Jul 13, 2026
wdberkeley
left a comment
wdberkeley
left a comment
Contributor
There was a problem hiding this comment.
Feel like there might be a black swan race hiding in all the timing coordination but I think it'd be unlikely enough we can merge and see if it ever happens 🤷
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a parameterized test for cloud topics and TSv2 which rapidly toggles the reconciler on and off while waiting for retention to be applied.
Asserts that after rounds of producing, waiting for retention to apply, and toggling the reconciler, that all data eventually makes it to L1 and it is consumable.
Backports Required
Release Notes