fix(ca): the coverage gate could not run, in any tree (#507)#512
Closed
SUaDtL wants to merge 1 commit into
Closed
Conversation
`tdd` Phase 5 and `refactor` Phase 2/6 instruct "run the coverage command from `tech-stack.md`", and both forbid guessing one. No coverage command existed - no provider installed in any of the three TypeScript trees, no script, and the word "coverage" appeared in tech-stack.md zero times. Every run reached the phase, found nothing to run, and passed through on a gap. A BLOCK gate that cannot execute is worse than an absent one: it reads as satisfied in every lane. Same defect class as #501's five suites that ran on nothing and #506's assertion that restated the bug. `refactor` was the worse casualty. Phase 2 is "Behavioral parity coverage proof" - the gate that justifies the entire lane by showing the pre-existing tests can detect a behavior change BEFORE production code is touched - and it rested wholly on a command that did not exist. The lane was running on Phase 5's "zero pre-existing tests modified" alone. The threshold table never named a metric, and that omission was load-bearing rather than cosmetic: a report gives four numbers that disagree, so "at least 70%" with no column named is not something anyone can be held to. ca-pi measured 85.37% lines against 78.73% branches - compliant at maturity 3, or not, depending purely on which column the reader picked. It is now lines AND branches, both binding. Lines catches code no test reaches at all, which is what #504 turned out to be: a `catch` with zero executions inside a 750-test suite. Branches catches the untaken half of a condition a test does reach. The threshold is deliberately NOT encoded in the three vitest configs. The skills apply it from CONTEXT.md `stage:` against maturity-coverage.md; copying the number into three package configs would fork the source of truth and the copies would drift the first time the stage moves. No CI job enforces coverage, also deliberately. `plugins/ca/tools` is below the stage-2 floor (67.22% lines, 59.46% branches), so wiring it into required CI would block every merge in the repo on an unrelated backfill. That backfill is tracked as its own issue rather than absorbed here, which would have turned #507 into "write several hundred statements of tests". A contract test now asserts that any command a gate reads from tech-stack.md is actually defined there. Verified against the pre-fix file rather than assumed: it fires. CHANGELOG: tdd Phase 5 and refactor Phase 2/6 can run their coverage gate; the maturity threshold now specifies lines and branches, both binding. Closes #507 Refs #511 Claude-Session: https://claude.ai/code/session_01JgdLb6n8mUdkFiKDTA37ML
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #507. Step 3 of the chain; stacked on #510 (base is
chore/vitest-4-ca-tools, so the diff stays clean — GitHub retargets tomainwhen #510 merges).The defect
tddPhase 5 andrefactorPhase 2/6 instruct "run the coverage command fromtech-stack.md", and both carry a hard rule forbidding guessing one. No coverage command existed — no provider in any of the three TypeScript trees, no script, andtech-stack.mdcontained the word "coverage" zero times.So every run reached the phase, found nothing to run, and passed through on a gap. A BLOCK gate that cannot execute is worse than an absent one — it reads as satisfied in every lane. Same defect class as #501 (five suites running on nothing) and #506 (an assertion that restated the bug).
refactorwas the worse casualty. Phase 2 is "Behavioral parity coverage proof" — the gate that justifies the entire lane by proving pre-existing tests can detect a behaviour change before production code is touched. It rested wholly on a command that did not exist, leaving the lane running on Phase 5's "zero pre-existing tests modified" alone.What lands
Tooling —
@vitest/coverage-v8@4.1.9in all three trees,npm run coveragein each, scope/provider/reporters in eachvitest.config.tsso the invocation is argument-free and platform-identical.ca/toolsgets its firstvitest.config.ts; coverage output is gitignored.The metric, named. The threshold table said only "minimum coverage". That omission was load-bearing, not cosmetic — a report gives four numbers that disagree:
plugins/ca/toolsplugins/ca-pi/toolsplugins/ca-sandbox/toolsca-pi is compliant at maturity 3 on lines and non-compliant on branches. "≥ 85%" with no column named is not a threshold anyone can be held to.
Now lines and branches, both binding. Lines catches code no test reaches — which is exactly what #504 turned out to be, a
catchwith zero executions inside a 750-test suite. Branches catches the untaken half of a condition a test does reach, which lines alone reports as covered. Statements duplicates lines; functions is noisiest at small counts.A guard against recurrence.
GateCommandTestasserts that any command a gate reads fromtech-stack.mdis actually defined there. Verified in both directions rather than assumed — run against the pre-fix file it fires, and it correctly identifies both demanders (tdd,refactor).Deliberate non-choices
CONTEXT.mdstage:againstmaturity-coverage.md. Copying the number into three package configs forks the source of truth, and the copies drift the first time the stage moves.ca/toolsis below the stage-2 floor, so wiring it into required CI would block every merge in the repo on an unrelated backfill. It is an orchestrator gate, which is what the skills already assume.stage:stays at 2. Raising to 3 today blocks all three trees — ca-pi fails branches at 78.73, ca-sandbox at 79.96. A gate nothing can pass gets overridden, and an overridden gate teaches readers it is noise.Versioning
The surface renders into three hosts, so this is a payload change for ca, ca-codex and ca-pi — not just a
tools/edit.Pi payload, version, changelog, and root metadata advanced together: 0.1.34 -> 0.1.35.v2.9.1/ca-codex-v0.3.0tag), and their gates permit a payload change on an unpublished version. Entry added under root## [Unreleased].Verification
npm run coveragegreen in all three trees; ca/tools still discovers all 324 tests after gaining a configunittest discoveroverplugins/ca/hooks/tests+check-plugin-refs.pysync-core.py --check,build-surface.py --check,build-host-packages.py --check— all in synctest_pi_package.py's exact workspace contract updated for the new script and dependencytsc --noEmitclean in all three treesbuild-surface --checkcaught me editing rendered files instead ofcore/surface/templates; the edits were moved to source and re-rendered.https://claude.ai/code/session_01JgdLb6n8mUdkFiKDTA37ML