Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .codearbiter/gate-events.log
Original file line number Diff line number Diff line change
Expand Up @@ -1005,3 +1005,19 @@ Claude-Session: https://claude.ai/code/session_015ZDVU1BzUqnnPVHbZ397bX') stages
[2026-07-26T17:57:51Z] REMIND [H-12] host=claude hook=post-write-edit.py | plugins/ca-pi/CHANGELOG.md is governed by ADR-0013-add-ca-pi-sibling-governance-plugin (Add ca-pi as a sibling governance plugin using the shared core and a thin Pi adapter). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T17:58:38Z] REMIND [H-12] host=claude hook=post-write-edit.py | plugins/ca-pi/tools/test/background-jobs.test.ts is governed by ADR-0013-add-ca-pi-sibling-governance-plugin (Add ca-pi as a sibling governance plugin using the shared core and a thin Pi adapter). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T17:58:38Z] REMIND [H-10] host=claude hook=post-write-edit.py | Possible hardcoded secret. Run the secret-handling check before committing. The commit will block until the gate records a pass.
[2026-07-26T19:29:01Z] REMIND [H-12] host=claude hook=post-write-edit.py | plugins/ca-pi/tools/vitest.config.ts is governed by ADR-0013-add-ca-pi-sibling-governance-plugin (Add ca-pi as a sibling governance plugin using the shared core and a thin Pi adapter). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:29:07Z] REMIND [H-12] host=claude hook=post-write-edit.py | plugins/ca-sandbox/tools/vitest.config.ts is governed by ADR-0007-second-plugin-ca-sandbox (Host a second sibling plugin (ca-sandbox) in the codeArbiter repo/marketplace). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:29:19Z] REMIND [H-07] host=claude hook=post-write-edit.py | Dependency manifest changed. Dispatch dependency-reviewer before committing (ORCHESTRATOR §5).
[2026-07-26T19:29:21Z] REMIND [H-12] host=claude hook=post-write-edit.py | plugins/ca-pi/tools/package.json is governed by ADR-0013-add-ca-pi-sibling-governance-plugin (Add ca-pi as a sibling governance plugin using the shared core and a thin Pi adapter). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:29:21Z] REMIND [H-07] host=claude hook=post-write-edit.py | Dependency manifest changed. Dispatch dependency-reviewer before committing (ORCHESTRATOR §5).
[2026-07-26T19:29:22Z] REMIND [H-12] host=claude hook=post-write-edit.py | plugins/ca-sandbox/tools/package.json is governed by ADR-0007-second-plugin-ca-sandbox (Host a second sibling plugin (ca-sandbox) in the codeArbiter repo/marketplace). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:29:22Z] REMIND [H-07] host=claude hook=post-write-edit.py | Dependency manifest changed. Dispatch dependency-reviewer before committing (ORCHESTRATOR §5).
[2026-07-26T19:34:42Z] REMIND [H-12] host=claude hook=post-write-edit.py | .codearbiter/tech-stack.md is governed by ADR-0001-hybrid-adr-living-docs-governance (Adopt a hybrid ADR + living-docs governance model). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:40:28Z] REMIND [H-12] host=claude hook=post-write-edit.py | core/surface/includes/maturity-coverage.md is governed by ADR-0011-multi-host-codex-plugin-shared-core (Multi-host support — third sibling plugin ca-codex via shared core + thin host adapters). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:40:35Z] REMIND [H-12] host=claude hook=post-write-edit.py | core/surface/skills/tdd/SKILL.md is governed by ADR-0011-multi-host-codex-plugin-shared-core (Multi-host support — third sibling plugin ca-codex via shared core + thin host adapters). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:40:37Z] REMIND [H-12] host=claude hook=post-write-edit.py | core/surface/skills/tdd/SKILL.md is governed by ADR-0011-multi-host-codex-plugin-shared-core (Multi-host support — third sibling plugin ca-codex via shared core + thin host adapters). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:40:45Z] REMIND [H-12] host=claude hook=post-write-edit.py | core/surface/skills/refactor/SKILL.md is governed by ADR-0011-multi-host-codex-plugin-shared-core (Multi-host support — third sibling plugin ca-codex via shared core + thin host adapters). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:40:46Z] REMIND [H-12] host=claude hook=post-write-edit.py | core/surface/skills/refactor/SKILL.md is governed by ADR-0011-multi-host-codex-plugin-shared-core (Multi-host support — third sibling plugin ca-codex via shared core + thin host adapters). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:41:40Z] REMIND [H-12] host=claude hook=post-write-edit.py | plugins/ca-pi/package.json is governed by ADR-0013-add-ca-pi-sibling-governance-plugin (Add ca-pi as a sibling governance plugin using the shared core and a thin Pi adapter). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
[2026-07-26T19:41:40Z] REMIND [H-07] host=claude hook=post-write-edit.py | Dependency manifest changed. Dispatch dependency-reviewer before committing (ORCHESTRATOR §5).
[2026-07-26T19:41:44Z] REMIND [H-12] host=claude hook=post-write-edit.py | plugins/ca-pi/CHANGELOG.md is governed by ADR-0013-add-ca-pi-sibling-governance-plugin (Add ca-pi as a sibling governance plugin using the shared core and a thin Pi adapter). If this change contradicts it, route to /ca:reconcile or /ca:adr — do not drift silently.
43 changes: 43 additions & 0 deletions .codearbiter/tech-stack.md
Original file line number Diff line number Diff line change
Expand Up @@ -141,6 +141,49 @@ scripts disabled. CI owns the Windows/macOS/Linux matrix.
`python -m py_compile plugins/ca/hooks/<file>.py` for any touched hook.
- TypeScript: `npm run typecheck` in `plugins/ca/tools` (only when tools changed).

## Coverage

One command per TypeScript tree, only when that tree changed:

```sh
npm --prefix plugins/ca/tools run coverage
npm --prefix plugins/ca-pi/tools run coverage
npm --prefix plugins/ca-sandbox/tools run coverage
```

Each prints a text summary and writes an html report to that tree's `coverage/`
(gitignored — run output, never project state). Scope, provider and reporters
live in each tree's `vitest.config.ts`; the script takes no arguments so it is
identical on every platform.

**The threshold is not encoded in the tooling.** `tdd` Phase 5 and `refactor`
Phase 2/6 apply it, reading `stage:` from `.codearbiter/CONTEXT.md` against
`plugins/ca/includes/maturity-coverage.md`. **Lines and branches must both
clear it**; a report satisfying one and not the other does not pass. Putting the
number in three `vitest.config.ts` files would fork that single source of truth
and the copies would drift the first time the stage moves.

Measured baseline at stage 2 (≥ 70%), 2026-07-26:

| tree | lines | branches | verdict |
| --- | --- | --- | --- |
| `plugins/ca/tools` | 67.22% | 59.46% | **below floor** — backfill tracked in #511 |
| `plugins/ca-pi/tools` | 85.37% | 78.73% | clears |
| `plugins/ca-sandbox/tools` | 86.13% | 79.96% | clears |

Two caveats when reading a local report:

- **ca-sandbox self-skips its docker-gated suites** on a host without Docker, so
a local number reads lower than required CI's. Compare against a run with
`CA_SANDBOX_REQUIRE_DOCKER=1` before concluding that tree regressed.
- **No CI job enforces coverage.** It is an orchestrator gate the skills run, not
a required check — deliberately, since wiring a red `ca/tools` into required CI
would block every merge on an unrelated backfill.

There is **no coverage tooling for the Python hooks**. `refactor` Phase 2 on a
Python surface therefore has no numeric floor to check; use the per-symbol
direct-test proof alone and say so in the phase record.

## Static checks (CI parity)

```sh
Expand Down
44 changes: 44 additions & 0 deletions .github/scripts/test_ci_impact.py
Original file line number Diff line number Diff line change
Expand Up @@ -2060,5 +2060,49 @@ def test_every_suite_is_reachable(self):
"these suites are invoked by no workflow and no sibling script, so "
"they never run: " + ", ".join(orphans))

class GateCommandTest(unittest.TestCase):
"""A gate that reads its command from tech-stack.md needs that command to exist.

Issue #507: `tdd` Phase 5 and `refactor` Phase 2/6 instructed "run the
coverage command from tech-stack.md", and tech-stack.md contained the word
"coverage" zero times. Both skills forbid guessing the command, so every
run reached the phase, found nothing to run, and passed through on a gap.
A BLOCK gate that cannot execute is worse than an absent one: it reads as
satisfied in every lane, which is the same failure shape as #501's suites
that ran on nothing and #506's assertion that agreed with the bug.

Enforced from THIS repo's own tech-stack.md, which is correct even though
the skills ship to other projects: the file is project state, so this
asserts that codeArbiter satisfies the contract its own gates impose.
"""

# token -> proof the command is actually defined, not merely discussed.
DEFINITIONS = {
"coverage": re.compile(r"(?m)^\s*npm\b.*\brun coverage\b"),
"typecheck": re.compile(r"(?m)^\s*npm\b.*\brun typecheck\b"),
}

def test_every_gate_command_read_from_tech_stack_is_defined_there(self):
tech_stack = (REPO_ROOT / ".codearbiter" / "tech-stack.md").read_text(encoding="utf-8")
skills = sorted((REPO_ROOT / "plugins" / "ca" / "skills").glob("*/SKILL.md"))
missing = []
for token, defined in sorted(self.DEFINITIONS.items()):
demanders = [
path.parent.name
for path in skills
if re.search(
rf"\b{token} command from `tech-stack\.md`",
path.read_text(encoding="utf-8"),
)
]
if demanders and defined.search(tech_stack) is None:
missing.append(
f"{token}: demanded by {', '.join(sorted(set(demanders)))}, "
f"but .codearbiter/tech-stack.md defines no such command")
self.assertEqual(
missing, [],
"a gate cannot run a command its tech-stack.md never defines: " + "; ".join(missing))


if __name__ == "__main__":
unittest.main()
9 changes: 9 additions & 0 deletions .github/scripts/test_pi_package.py
Original file line number Diff line number Diff line change
Expand Up @@ -914,12 +914,21 @@ def test_build_workspace_is_isolated_and_exactly_pinned(self):
"build": "node ./build.mjs",
"typecheck": "tsc --noEmit",
"test": "vitest run",
# Issue #507: tdd Phase 5 and refactor Phase 2/6 read a coverage
# command from tech-stack.md and forbid guessing one. Before this
# script existed there was nothing to read, so both gates passed
# on a gap. Scope and reporters live in vitest.config.ts so the
# invocation stays argument-free and identical on every platform.
"coverage": "vitest run --coverage",
},
)
self.assertEqual(
data["devDependencies"],
{
"@types/node": "25.9.4",
# Peer-pinned to vitest EXACTLY by upstream; the two must move
# together or `npm ci` ERESOLVEs.
"@vitest/coverage-v8": "4.1.9",
"esbuild": "0.28.1",
"typescript": "5.9.3",
"vitest": "4.1.9",
Expand Down
6 changes: 6 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,12 @@ plugins/ca-sandbox/tools/node_modules/
plugins/ca-sandbox/tools/**/.nixpacks/
plugins/ca-sandbox/tools/**/.ca-sandbox.nixpacks.Dockerfile

# Coverage output (issue #507). `npm run coverage` in any plugins/*/tools tree
# writes a text summary to stdout and an html report here. Run output, never
# project state — the maturity floor is applied by the tdd/refactor skills from
# the reported numbers, not from a committed report.
plugins/*/tools/coverage/

# ca-sandbox real-container execution sentinel (issue #406) — the append-only
# record of which docker-gated layers actually ran. Written per-run by
# docker-gate.ts when CA_SANDBOX_DOCKER_SENTINEL is set (required CI does; a
Expand Down
27 changes: 27 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,33 @@ predate the plugin rewrite and are grouped by date.

## [Unreleased]

### Fixed

- `tdd` Phase 5 and `refactor` Phase 2/6 can actually run. Both instructed
"run the coverage command from `tech-stack.md`" and both forbid guessing one -
and no coverage command existed, in any tree, for any of the four plugins. So
every run reached the phase, found nothing to run, and passed through on a
gap. A BLOCK gate that cannot execute is worse than an absent one: it reads as
satisfied in every lane, which is the same defect class as issue #501's five
suites that ran on nothing (#507).

`refactor` was the worse casualty. Phase 2 is "Behavioral parity coverage
proof" - the gate that justifies the lane by showing the tests can detect a
behavior change BEFORE production code is touched - and it rested entirely on
a command that did not exist.

- The threshold table never said WHICH metric, and the omission was
load-bearing rather than cosmetic: a report gives four numbers that disagree,
so "≥ 70%" with no column named is not something anyone can be held to. It is
now **lines and branches, both binding**. Lines catches code no test reaches
(issue #504 was exactly that - a `catch` with zero executions inside a
750-test suite); branches catches the untaken half of a condition a test does
reach, which lines alone reports as covered.

- A contract test now asserts that any command a gate reads from
`tech-stack.md` is actually defined there, so the next gate cannot ship
pointing at nothing. Verified against the pre-fix file: it fires.

### Added

- `/ca:review` can review an inbound GitHub pull request, not only the diff you
Expand Down
26 changes: 26 additions & 0 deletions core/surface/includes/maturity-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,29 @@ The single source of truth for the coverage threshold. Coverage scales with the
| 2 | ≥ 70% |
| 3 | ≥ 85% |
| 4 | ≥ 90% |

## Which metric (issue #507)

**Lines and branches. Both must clear the threshold.** A report satisfying one and not the other
does not pass.

This was previously unstated, and the omission was load-bearing rather than cosmetic: a coverage
report gives four numbers that disagree, so "≥ 70%" without a column named is not a threshold anyone
can be held to. Measured on codeArbiter itself at the time of writing, one tree sat at 85.37% lines
and 78.73% branches — compliant at maturity 3, or not, depending purely on which column the reader
picked.

- **Lines** catches code no test reaches at all — a `catch` block with zero executions inside a
passing suite, which no assertion is ever going to surface.
- **Branches** catches the untaken half of a condition a test does reach: the error arm of an `if`,
the fallback of a `??`. Line coverage alone reports those as covered.
- **Statements** duplicates lines closely enough to add nothing. **Functions** is the noisiest
column at small counts, where one uncovered helper moves it several points.

**The number is the floor, not the goal.** A test written only to move a percentage is worse than
the gap it closed, because it converts an honest red into a green that asserts nothing. When
backfilling to clear this bar, work the uncovered *report* — error and refusal paths first — and let
the number follow.

Where a surface has no coverage tooling at all, there is no numeric floor to check. Record that
explicitly; do not invent a command, and do not treat the phase as passed unexamined.
11 changes: 8 additions & 3 deletions core/surface/skills/refactor/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,9 +39,14 @@ using the shared threshold table `{{PLUGIN_ROOT}}/includes/maturity-coverage.md`

Every public method in the surface table MUST have at least one direct test — transitive coverage through a higher-level integration test does not count. A public method with zero direct tests is uncovered for this gate.

If surface coverage is below the maturity threshold, OR any public method has zero direct tests, halt and route to the `tdd` skill Phase 1 to backfill obligations and red tests for the uncovered surface. Resume Phase 2 only after the backfill is green.
**Lines and branches must both clear the threshold** (issue #507); a surface satisfying one and not
the other is not proven. Where the surface has no coverage tooling, the per-symbol direct-test proof
stands alone: record that there is no numeric floor for this surface rather than inventing a command
or treating the phase as passed unexamined.

Gate: surface coverage at or above the maturity threshold AND every public method backed by a direct test. Otherwise backfill via `tdd` Phase 1 before retrying.
If surface coverage is below the maturity threshold on either metric, OR any public method has zero direct tests, halt and route to the `tdd` skill Phase 1 to backfill obligations and red tests for the uncovered surface. Resume Phase 2 only after the backfill is green.

Gate: surface coverage at or above the maturity threshold on BOTH lines and branches AND every public method backed by a direct test. Otherwise backfill via `tdd` Phase 1 before retrying.

## Phase 3 — Red parity tests (conditional) · gate: BLOCK

Expand All @@ -67,7 +72,7 @@ Gate: full suite green with zero pre-existing tests modified. BLOCK if any pre-e

## Phase 6 — Lint and coverage · gate: BLOCK

Run lint, the type-check if the project is statically typed, and coverage, all from `tech-stack.md`. Resolve every lint and type error. Confirm surface coverage remains at or above the maturity threshold — a refactor MUST NOT reduce coverage of the surface it touched.
Run lint, the type-check if the project is statically typed, and coverage, all from `tech-stack.md`. Resolve every lint and type error. Confirm surface coverage remains at or above the maturity threshold on both lines and branches — a refactor MUST NOT reduce coverage of the surface it touched on either metric.

Gate: clean lint and type-check, zero errors, and no coverage regression on the named surface. "Mostly passes" is not passing — this is what clears the path to `commit-gate`.

Expand Down
13 changes: 11 additions & 2 deletions core/surface/skills/tdd/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,13 +84,22 @@ Coverage scales with the maturity value (`stage:` in `CONTEXT.md`) — a rigor k
gate. The threshold table is the shared `{{PLUGIN_ROOT}}/includes/maturity-coverage.md` (the
single source of truth, also used by `refactor` Phase 2).

Run the coverage command from `tech-stack.md`. Below the maturity threshold → add tests until it is met.
Run the coverage command from `tech-stack.md`. **Lines and branches must both clear the threshold**
— a report satisfying one and not the other does not pass (issue #507). Below it on either → add
tests until both are met.

Where the surface has no coverage tooling at all, there is no numeric floor to check. Record that
explicitly and pass the phase on the Phase 4 obligation verify alone; do NOT invent a command, and
do NOT silently skip the phase as though it had been run — a gate that cannot execute still reads as
satisfied, which is worse than an absent one.

**Stakes:** when coverage blocks below threshold, name the class of code left dark — the paths a later
regression could rot unnoticed — not just "below threshold." The number is the rule; the untested paths
are why it matters.

Gate: threshold met for the current maturity value.
Gate: threshold met on BOTH lines and branches for the current maturity value, or the surface
recorded as having no coverage tooling. A test added only to move the percentage fails this gate in
spirit — it converts an honest red into a green that asserts nothing.

## Phase 6 — Lint · gate: BLOCK

Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "ca-pi",
"version": "0.1.34",
"version": "0.1.35",
"private": true,
"license": "AGPL-3.0-only",
"engines": {
Expand Down
Loading