Skip to content

test: cover every DSL shape family in the N-1 upgrade lifecycle - #411

Merged
Pino de Candia (pinodeca) merged 2 commits into
mainfrom
test/upgrade-lifecycle-shape-families
Sep 30, 2026
Merged

Pino de Candia (pinodeca) merged 2 commits into
mainfrom
test/upgrade-lifecycle-shape-families

Conversation

@pinodeca

@pinodeca Pino de Candia (pinodeca) commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Extend the real N-1 binary upgrade lifecycle to cover all seven behavior families
represented by the fixed DSL shape corpus. Each family has an instance that
suspends inside its structure and resumes under B1 (new binary / old schema),
and another that resumes under B2 (new binary / new schema).

The lifecycle also covers a sequence created by the new binary against the old
schema, suspended across ALTER EXTENSION UPDATE, and resumed on the new schema.
Fresh completed sequences in each phase verify new execution and stable access
to previously completed results. The final phase validates 18 instances.

Coverage

Family Suspension point Required behavior
seq Between two sequenced markers Ordered continuation, captured variables, and runtime {sys_label} substitution
if-then Inside the taken then-branch Then continuation runs; else stays unrun
if-else Inside the taken else-branch Else continuation runs; then stays unrun
loop During the first iteration Remaining iterations produce exact body and counter marker counts
break During logical iteration 1, before breaking on iteration 3 Independent captured iteration state and marker values [1, 2, 3]
join One branch done, one suspended Completed branch is not re-executed on resume
race Winner suspended, loser on a durable long timer Winner completes; exact loser child reaches terminal cancellation without marking

df.sql is exercised by every family.

Upgrade phases

  • Baseline, N-1 binary / N-1 schema: suspend two instances per family and
    complete one sequence.
  • B1, N binary / N-1 schema: restart with the candidate binary, revalidate
    existing instances, resume each family's first instance, complete a fresh
    sequence, and suspend seq-binary-b2.
  • B2, N binary / N schema: apply ALTER EXTENSION UPDATE, revalidate, resume
    each family's second instance and seq-binary-b2, and complete a fresh sequence.

Checks before either resume establish that the instances are still suspended
and their signal subscriptions are durable.

Assertions

  • Exact path-tagged marker counts before and after resume, including no effects
    from untaken branches or the race loser; marker execution must use the granted
    ordinary role.
  • Sequence seed markers equal 41 before and after resume, continuations equal
    42, and completed outputs remain byte-identical across later phases despite
    changes to live variables. Both sequence markers resolve {sys_label} at runtime.
  • Break control uses a captured logical counter independent of marker counts.
    The expected values are [1] while suspended and [1, 2, 3] after completion,
    so a duplicate effect cannot advance the break condition and conceal a missing
    iteration.
  • Race validation identifies the exact loser child from the race node and parent
    execution. It requires a durable TimerCreated and running status before
    resume, then a terminal failed execution with an application Cancelled error.
    An absent loser marker alone is not considered proof of cancellation.
  • Signal subscriptions are checked in root histories and spawned branch histories.
  • df.status, df.result, df.list_instances, df.instance_info, and provider
    state are queried as an ordinary granted role.
  • Harness unit tests cover invalid marker counts and values, missing subscriptions,
    absent timers or children, non-cancellation failures, delayed loser cancellation,
    phase ordering, exact instance cohorts, and variable capture.

Scope and limitations

This is curated N-1 replay coverage, not a historical release chain or a transplant
of all 154 fixed-corpus cases. It covers one suspended instance per family at each
upgrade boundary, plus the new-binary/old-schema sequence cohort, rather than
every nested permutation. Race losers use 3,600-second timers to keep the intended
winner suspended safely during the upgrade.

Validation

  • Real 0.2.8 -> 0.2.9 lifecycle: all five checkpoints pass, with 18 instances
    validated in the final phase.
  • Fault injection against the real lifecycle: a duplicated pre-suspension break
    marker produces [1, 1, 2, 3] and is rejected while all three logical
    iterations still execute.
  • python3 -m unittest discover -s scripts -p test_upgrade_lifecycle.py:
    25 tests passed.
  • cargo build --features pg17.
  • cargo clippy --features pg17.
  • ./scripts/test-unit.sh: 424 passed, 16 ignored.
  • ./scripts/test-e2e-local.sh: 65 passed across all required PostgreSQL modes.
  • cargo fmt -p pg_durable -- --check and git diff --check.

Upgrade & Migration

Test infrastructure and documentation only. No production Rust, extension DDL/API,
version, dependency, or upgrade-script changes. No additional runtime schema
detection is required. The B1 supported-schema contract is unchanged; the existing
Guarantee A, all-supported-schema B1 matrix, and B2 catalog/grant checks remain
in place.

Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com

Extend the real previous-binary upgrade lifecycle so every behavior family
introduced by the fixed shape corpus (#410) has an N-1-created instance that
suspends inside its structure and resumes under both the B1 (new binary / old
schema) and B2 (new binary / new schema) boundaries. This adds in-flight replay
evidence for sequences, conditionals (then and else), loops, break-loops,
joins, and races, rather than transplanting all 154 corpus cases.

Each family records path-tagged marker rows keyed by (label, path, occurrence).
The harness asserts exact per-path counts before and after resume, including
zero-count paths for unrun branches and cancelled race losers, detecting
missing, duplicate, reordered, or incorrectly selected work when an old history
replays. The seq family additionally verifies captured-variable reuse and
byte-identical completed results across phases. The durable-subscription check
now also matches join/race branches, whose wait_for_signal subscribes in a
spawned child sub-orchestration prefixed with the root instance id.

Validated against the real 0.2.8 -> 0.2.9 lifecycle (all five phases, 17
instances) and 17 harness unit tests.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Verify independent break iterations and terminal race loser cancellation. Retain the new-binary/old-schema cohort across ALTER EXTENSION UPDATE, runtime labels, and explicit sequence seed assertions.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@pinodeca
Pino de Candia (pinodeca) merged commit 1e8d254 into main Sep 30, 2026
9 checks passed
@pinodeca
Pino de Candia (pinodeca) deleted the test/upgrade-lifecycle-shape-families branch September 30, 2026 21:00
Pino de Candia (pinodeca) pushed a commit that referenced this pull request Oct 1, 2026
Prove the real previous-binary upgrade lifecycle (#409, #411) still catches
replay-incompatible changes that fresh-execution unit and E2E tests miss, and
document the compatibility contract so contributors do not break duroxide
replay.

- scripts/upgrade_lifecycle_canary.py: negative controls that mutate a
  throwaway git worktree and require scripts/upgrade_lifecycle.py to fail with a
  duroxide schedule mismatch after the N-1 baseline validates. Two canaries pin
  the two coverage layers: a changed execute_sql activity input (real N-1
  histories, #409) and reversed JOIN branch scheduling with restored output
  (per-family suspension, #411).
- scripts/test_upgrade_lifecycle_canary.py: fast unit tests for the pure logic
  and anchor integrity, so anchor drift fails in milliseconds, not after a build.
- ci.yml: run the canary harness unit tests in required CI.
- upgrade-canary.yml: scheduled/manual job for the expensive end-to-end canary;
  the ordinary lifecycle stays the required per-PR gate.
- docs/upgrade-testing.md: document the canaries and add a canonical
  "Coverage boundaries and known gaps" matrix.
- CONTRIBUTING.md, copilot-instructions.md: a replay-compatibility checklist and
  strengthened determinism guidance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pino de Candia (pinodeca) pushed a commit that referenced this pull request Oct 1, 2026
Prove the real previous-binary upgrade lifecycle (#409, #411) still catches
replay-incompatible changes that fresh-execution unit and E2E tests miss, and
document the compatibility contract so contributors do not break duroxide
replay.

- scripts/upgrade_lifecycle_canary.py: negative controls that mutate a
  throwaway git worktree and require scripts/upgrade_lifecycle.py to fail with a
  duroxide schedule mismatch after the N-1 baseline validates. Two canaries pin
  the two coverage layers: a changed execute_sql activity input (real N-1
  histories, #409) and reversed JOIN branch scheduling with restored output
  (per-family suspension, #411).
- scripts/test_upgrade_lifecycle_canary.py: fast unit tests for the pure logic
  and anchor integrity, so anchor drift fails in milliseconds, not after a build.
- ci.yml: run the canary harness unit tests in required CI.
- upgrade-canary.yml: scheduled/manual job for the expensive end-to-end canary;
  the ordinary lifecycle stays the required per-PR gate.
- docs/upgrade-testing.md: document the canaries and add a canonical
  "Coverage boundaries and known gaps" matrix.
- CONTRIBUTING.md, copilot-instructions.md: a replay-compatibility checklist and
  strengthened determinism guidance.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant