Repository navigation
Conversation
…oralio#12966) Problem: Deleted deployment versions accumulate on passive clusters. They can block new builds after failover and eventually make replication updates too large to store or transmit. 1. **Normal deletion — the case this PR addresses** **Setup:** Active and passive start with `{A, B}`, clock 10. Active deletes A and sends `{B}`, clock 20. **Before:** The passive unions both snapshots and retains `{A, B}`. **After:** The passive replaces its deployment data with `{B}`. An equal incoming clock also replaces, allowing force replication to clear existing accumulation. 2. **Delayed replication — fixed by [temporalio#12351](temporalio#12351), which introduced the normal-deletion issue this PR addresses** **Setup:** After failover, the target has `{B}`, clock 20. A delayed snapshot from the old active contains `{A}`, clock 10. **Before:** The union fix merged to `{A, B}`, preserving valid A despite its older parent clock. **After:** Still merges to `{A, B}`. This change retains the original fix's protection for delayed replication. 3. **Accepted edge case** **Setup:** The target has valid `{B}`, clock 20, but a delayed incoming snapshot contains only `{A}`, clock 30, for example because of clock skew. **Before:** Union preserves `{A, B}`. **After:** Replacement produces `{A}`, losing valid B. This remains an accepted limitation outside normal single-active clock ordering; this fix covers routine deletions and the older-incoming race in example 2. **Rollback protection:** `matching.enableTaskQueueUserDataRevisionMerge` defaults to `true`. If this fix causes issues, turn it off to restore the parent-clock snapshot selection used before [temporalio#12351](temporalio#12351). This also gives up the delayed-replication protection in example 2, so we would need to revisit a different solution. Missing clocks retain compatibility merging while enabled. ### Testing Unit tests; replication tests also run with Go's race detector. (cherry picked from commit e4160f8)
Contributor
|
Claude encountered an error after 0s —— View job I'll analyze this and get back to you. |
thearcticwatch
approved these changes
Nov 6, 2026
simvlad
approved these changes
Nov 6, 2026
rkannan82
enabled auto-merge (squash)
November 6, 2026 14:29
yiminc
approved these changes
Nov 6, 2026
stpierre
approved these changes
Nov 6, 2026
rkannan82
merged commit Nov 6, 2026
159e025
into
temporalio:cloud/v1.33.0-165
84 of 86 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed?
Backports #12966 to Cloud 165. If active deletes A from
{A, B}, a newer or equal-clock incoming snapshot now replaces passive deployment data with{B}, removing the orphan. Older incoming snapshots and missing clocks still merge by revision.matching.enableTaskQueueUserDataRevisionMergedefaults totrue. Disabling it restores parent-clock snapshot selection from before the original union fix. An equal or newer incoming snapshot can still remove valid target-only data; this is the same accepted limitation as the main PR.Release-only union-preservation tests are updated to retain their older/missing-source-clock cases. The backported tests cover equal/newer replacement and missing-target-clock merging.
Original PRs
Why is this necessary as a patch?
The unconditional union merge retains versions deleted on the active. These accumulate on passive clusters and can block new worker builds after failover when the task queue version limit is exceeded.
What testing or retesting is appropriate for this patch after merge?
Matching unit tests and focused replication tests with Go's race detector. Run the standard patch validation.