diff --git a/docs/plans/2026-08-14-001-refactor-harness-identity-and-release-validation-plan.md b/docs/plans/2026-08-14-001-refactor-harness-identity-and-release-validation-plan.md new file mode 100644 index 0000000..3877e6b --- /dev/null +++ b/docs/plans/2026-08-14-001-refactor-harness-identity-and-release-validation-plan.md @@ -0,0 +1,322 @@ +--- +title: "Harness Identity and Parsed Release Validation - Plan" +type: refactor +date: 2026-08-14 +artifact_contract: "ce-unified-plan/v1" +artifact_readiness: implementation-ready +product_contract_source: ce-plan-bootstrap +execution: code +--- + +# Harness Identity and Parsed Release Validation - Plan + +## Goal Capsule + +- **Objective:** Two behavior-preserving refactors of the `scripts/` layer: (1) a new `scripts/harness-identity.ts` leaf module that becomes the single source of harness vocabulary, paths, and driver seams, with all TypeScript branch sites migrated to it; (2) `scripts/release-validate.ts` workflow checks converted from literal substring greps to `Bun.YAML.parse` structural assertions under a strict parity ledger. +- **Authority:** This plan's Requirements and KTDs govern scope. ADR-0001 (per-harness manifests and hook declarations) and the persisted proof/receipt contracts are hard constraints — no unit may violate them. +- **Execution profile:** Two independent PRs. PR 1 = Work 2 (release validation, U1–U4). PR 2 = Work 1 (harness identity, U5–U11), rebased onto PR 1 (only overlap: `scripts/release-validate.ts:526` hooks-path guard). U11 (driver extraction) is droppable to follow-up without unwinding anything. +- **Stop conditions:** Stop and surface if: a parity-ledger literal cannot be classified into a tier without changing what the validator guarantees; generated `plugin/` bytes change under `generate:check`; any persisted JSON or receipt-filename contract would change; or an asserted error message must change to proceed. +- **Tail ownership:** Executor owns branch/PR mechanics per repo conventions (PRs from the myagentdojo account; hosted-canary approval expected per PR since both touch `scripts/`). + +--- + +## Product Contract + +### Summary + +Create `scripts/harness-identity.ts` as the one owner of harness identity knowledge and migrate all TypeScript branch sites to it, adding validation-only parity checks for the shell hook and workflow surfaces that cannot import it. Separately, rewrite `scripts/release-validate.ts`'s release-workflow checks to navigate parsed YAML structure instead of grepping raw text, holding strict parity via a three-tier ledger. + +### Problem Frame + +An architecture review (re-verified 2026-08-14 at f06ad99) found harness knowledge re-derived at ~70 sites across 12 script files with three competing vocabularies, and the gap widening — `scripts/codex-production-update.ts` added 13 more branch sites since the review. The same hooks-path pair is written four independent times. Separately, `release-validate.ts` asserts 58 literal substrings against `.github/workflows/release.yml`: whitespace changes break it while policy is intact, and a semantically broken workflow containing the substring passes. The repo already demonstrates the remedy for both: `scripts/prove-harness-install.ts:1370` derives hooks paths from one vocabulary, and `scripts/repository-readiness.ts:733` parses workflows with `Bun.YAML.parse`, fail-closed. + +### Key Decisions + +- **Work architecture candidates 1 and 2; defer the rest** (session-settled: user-directed — chosen over candidates 3–5 and the standalone test findings from the same review: highest leverage, and candidate 1 is the only module already named in the glossary with no code). Governs R1–R12. +- **Full migration of all TypeScript branch sites** (session-settled: user-directed — chosen over adapter-plus-partial-migration: the gap measurably widened while two patterns coexisted). Governs R2, R3. +- **Validation-only parity for non-TypeScript surfaces** (session-settled: user-directed — chosen over code generation: kills silent drift at a fraction of generation's cost; the hook entrypoint is POSIX shell and can never import TypeScript). Governs R5. +- **Strict parity for release checks** (session-settled: user-directed — chosen over semantic restatement: mechanism change and policy change must not share a diff; these checks guard the release path). Governs R8, R9. +- **Persisted contracts are frozen** (session-settled: user-directed — chosen over migrating the `-cli` proof/receipt vocabulary to canonical IDs: rewriting persisted formats turns a refactor into a data migration with external blast radius). Governs R4. + +### Requirements + +**Harness identity (Work 1)** + +- R1. A single module owns harness identity: canonical IDs, display names, per-harness paths (hooks declaration, manifest directory), the plugin-root env-var mapping, and the qualification-client vocabulary. +- R2. Every TypeScript branch site that discriminates on harness identity reads from that module; no script defines its own harness union, path template, or display mapping. +- R3. Behavior is preserved: generated `plugin/` output is byte-identical, all asserted error messages are byte-identical, and every persisted JSON shape is unchanged. +- R4. The qualification-client vocabulary (`claude-cli`, `codex-cli`, `codex-desktop`) is modelled as a distinct type mapped from canonical IDs; its persisted surfaces (proof JSON `client` fields, receipt filename convention) are unchanged. `codex-desktop` gains no code path — it exists as a vocabulary value only. +- R5. The shell hook (`plugin/hooks/native-capability-hook` case arms) and the pinned CLI-install line shared by `plugin-ci.yml`, `hosted-canary.yml`, and `release.yml` are covered by parity tests that fail when they diverge from the module's values. +- R6. `CONTEXT.md` gains a glossary entry for the new term; a new ADR records the vocabulary decision and the driver-seam shape; `docs/agents/doc-targets.yml` gains rows binding both docs to their verifying artifacts. +- R7. The Claude install driver is extracted behind a dependency-injection interface following the same pattern as `CodexDriverDependencies` (same lifecycle shape; harness-typed state payloads; not the same method surface). + +**Release validation (Work 2)** + +- R8. Workflow assertions in `release-validate.ts` navigate parsed YAML structure: job existence and order, step navigation, `env`/`permissions`/`concurrency`/`uses`/`needs` asserted as typed values, action pins asserted by walking `jobs.*.steps[].uses`. +- R9. Every current literal is accounted for in a parity ledger with a tier and comparison mode: structural, step-scoped run substring, raw residual (comment-satisfied and forbidden-anywhere checks), or recorded drop with reason. No literal silently disappears. +- R10. Unparseable `release.yml` fails closed: exit 1 with a clear message (new coverage — no test exercises this today). +- R11. Behavior is preserved for consumers: exit codes, stderr messages asserted by `scripts/release-validate.test.ts`, and the `--repair` surface are unchanged. + +**Cross-cutting** + +- R12. The full suite passes after every implementation unit; `generate:check` stays clean throughout. + +### Scope Boundaries + +- The generated Plugin Payload does not change. Each Harness keeps its own manifest and hook declarations per ADR-0001 — this work centralizes the derivation of the two distinct paths, never the paths themselves. +- Prose mentions of "Claude"/"Codex" in error messages and help text do not migrate; only branch discriminators, type unions, and mapping call sites do. A "no orphan union" parity rule applies to discriminators only. +- `scripts/ship-canary.ts` and `scripts/repository-readiness.ts` contain no harness-vocabulary branch sites and are untouched. + +#### Deferred to Follow-Up Work + +- Architecture candidates 3 (subprocess seam), 4 (candidate lineage), 5 (prove-* shape) and the review's three standalone test findings. +- Generating the shell hook / workflow harness fragments from the module (upgrade path if validation-only parity ever proves insufficient). +- Consolidating the six existing hardcoded pin-version assertions in `release-validate.test.ts` and `ship-canary.test.ts` into the new version-agnostic parity test. + +--- + +## Planning Contract + +### Key Technical Decisions + +- KTD1. **Canonical harness ID is lowercase `"claude" | "codex"`** (session-settled: user-directed — chosen over the `-cli` and capitalized vocabularies: lowercase already keys real filesystem layout and is the dominant form). The capitalized form survives only as a display mapping (it is part of `HarnessInstallRecoveryError`'s asserted message contract); `dev.ts`'s local `type Harness` is deleted in favor of the module's type. +- KTD2. **The registry is two explicit per-harness records, never derivation templates.** The env-var mapping is asymmetric (`claude → CLAUDE_PLUGIN_ROOT`, `codex → PLUGIN_ROOT`, not `CODEX_PLUGIN_ROOT`); a `toUpperCase()` template would silently change generated bytes. Qualification clients are a separate three-value type with a client→harness mapping, because `codex` maps to two clients — it cannot be a per-harness field. +- KTD3. **`harness-identity.ts` is a leaf module**: it imports no sibling scripts. `plugin-config.ts → harness-identity` is the safe direction; the reverse is forbidden. It must never import the six side-effectful entry scripts (`generate.ts`, `package.ts`, `init.ts`, `prove-distribution.ts`, `prove-dx.ts`, `prove-runtime-custody.ts`). +- KTD4. **Expected values come from canonical owners, not re-encoded literals.** Parity tests and structural assertions read from `harness-identity.ts` / `plugin-config.ts` values wherever an owner exists; re-encoding expected values as fresh literals would swap string drift for constant drift. The workflow pin-line parity test asserts the three workflows agree with each other byte-for-byte without knowing the version, so routine CLI bumps do not touch the module. +- KTD5. **Three-tier parity ledger for Work 2** (session-settled: user-approved — step-scoped leaf checks chosen over whole-file greps and over attempting to parse shell). Tiers: (1) structural — typed-value assertions on parsed YAML, including boolean coercion (`overwrite: true` parses as boolean) and expression-contains semantics where a literal is a substring of a larger `if:` expression; (2) step-scoped run substrings — shell fragments checked inside their owning step's parsed `run` string (block scalars strip indentation, so checks target parsed values, not raw slices); where the owning step is unnamed, scope to the job's concatenated run strings and record the weaker scope in the ledger; (3) raw residual — checks only raw text can express: the comment-satisfied `skip-github-release` literal (re-anchor to the existing `release-please-config.json` check at `release-validate.ts:569` and record the comment literal as a drop) and the forbidden-anywhere negatives (`parent_count`, `mergeMode`, `github.run_attempt`), which must scan comments too. The ledger lives in code next to the assertions, one entry per current literal: tier, owner (job/step/field path), comparison mode, or drop reason. +- KTD6. **Job order via `Object.keys` insertion order; fail-closed parse.** Bun.YAML (bun 1.3.14) preserves key insertion order and parses `on:` as the string key `"on"`. Unparseable YAML exits 1 with a message (adapted from `repository-readiness.ts:733-741`'s fail-closed shape, which returns a classification — the validator must exit instead). +- KTD7. **Error messages byte-identical.** `release-validate.test.ts` asserts stderr substrings and mutation tests depend on exact messages; Work 1's migration likewise preserves every thrown message. Cheapest honest behavior-preservation proof. +- KTD8. **Mutation anchors guarded.** Each mutation helper in `release-validate.test.ts` asserts `mutated !== original` before writing, so structural validation cannot silently turn mutation tests into no-ops. +- KTD9. **Claude driver gets a Claude-specific DI interface** (session-settled: user-directed — extraction chosen over leaving the inlined driver, as a droppable final unit). Same DI pattern as `CodexDriverDependencies`, not the same shape: the Claude driver's surface differs (three-scope loop, `findClaudeInstall`/`replaceClaudeInstall`/`claudeEnvironment` shared with `proveHostedHarnessInstall`). Shared lifecycle shape (preflight → capture → mutate → verify → restore-on-failure) with harness-typed state payloads, never a unioned common struct — the new ADR states this constraint natively and cross-references ADR-0001 and ADR-0003. Existing type-only back-import pattern is preserved or improved by moving shared install-state types to a neutral module; the 12 named exports `prove-harness-install.test.ts` imports stay stable. +- KTD10. **Two PRs, Work 2 first** (session-settled: user-directed — chosen over one combined PR: Work 2 is self-contained and carries the heaviest `release-validate.ts` churn; Work 1 rebases onto it touching only the hooks-path guard at `:526`). +- KTD11. **Trust-but-verify the suite** (session-settled: user-approved — chosen over a characterization-test layer: the suite is 596 tests at 1.57× test:source). During implementation, the four hooks-path assertion sites get a deliberate mutation check — change the derived path, confirm a test fails — before the suite is trusted over them. A Test Design Brief precedes every test-artifact change. + +### High-Level Technical Design + +Module topology after Work 1 — `harness-identity` is a leaf; arrows point from consumer to owner: + +```mermaid +flowchart TB + subgraph identity["scripts/harness-identity.ts (new leaf)"] + HID["type HarnessId = claude | codex
per-harness records: hooksPath, manifestDir,
pluginRootEnvVar, displayName"] + QC["type QualificationClient = claude-cli |
codex-cli | codex-desktop
client→harness mapping"] + end + PC["plugin-config.ts
(generates manifests + hooks.json)"] --> HID + BUILD["build.ts"] --> HID + DEV["dev.ts"] --> HID + PDX["prove-dx.ts"] --> HID + RV["release-validate.ts"] --> HID + PHI["prove-harness-install.ts"] --> HID + PHI --> QC + HIR["harness-install-recovery.ts
(displayName mapping)"] --> HID + HIC["harness-install-codex.ts"] --> HID + UPD["update.ts / prove-distribution.ts /
codex-production-update.ts"] --> HID + PT["parity tests
(shell-hook case arms,
workflow pin line)"] -.read values.-> HID + GEN["plugin/hooks/{claude,codex}/hooks.json
+ manifests (generated, per ADR-0001)"] + PC -->|generate:check byte-identical| GEN +``` + +Work 2 — tier routing for each of the ~58 current literals: + +```mermaid +flowchart TB + L["current literal"] --> D1{"expressible as a typed value
on parsed YAML?"} + D1 -->|yes| T1["Tier 1: structural assertion
(job/step/field path, typed compare,
expression-contains for if: substrings)"] + D1 -->|no| D2{"shell fragment inside
a run: block?"} + D2 -->|yes| T2["Tier 2: substring scoped to owning
step's parsed run string
(job-scoped if step unnamed — recorded)"] + D2 -->|no| D3{"only raw text can express it?
(comment-satisfied, forbidden-anywhere)"} + D3 -->|yes| T3["Tier 3: raw residual check"] + D3 -->|no| DROP["recorded drop with reason
(e.g. re-anchored to config owner)"] + T1 & T2 & T3 & DROP --> LEDGER["parity ledger entry
(one per literal — R9)"] +``` + +### Assumptions + +- Hosted-canary environment approval is granted per PR (both touch `scripts/`, which `ship-canary.ts:100` classifies as publishing-system). +- Bun stays at the pinned 1.3.14 behavior for YAML key order; U1's ledger work re-verifies before relying on it. + +--- + +## Implementation Units + +Unit index: + +| U-ID | Title | Key files | Depends on | +|---|---|---|---| +| U1 | Parity ledger + fail-closed test | `scripts/release-validate.ts`, `scripts/release-validate.test.ts` | — | +| U2 | Structural navigation core | `scripts/release-validate.ts` | U1 | +| U3 | Migrate assertions to tiers | `scripts/release-validate.ts` | U2 | +| U4 | Mutation-anchor guards | `scripts/release-validate.test.ts` | U3 | +| U5 | `harness-identity.ts` module | `scripts/harness-identity.ts` (+test) | — | +| U6 | Migrate generators | `scripts/plugin-config.ts` | U5 | +| U7 | Migrate validators + proofs | `build.ts`, `prove-dx.ts`, `release-validate.ts`, `prove-harness-install.ts`, `dev.ts`, `update.ts`, `prove-distribution.ts` | U5, U6, U3 | +| U8 | Display vocabulary migration | `harness-install-recovery.ts`, `harness-install-codex.ts`, `codex-production-update.ts` | U5 | +| U9 | Non-TS parity tests | `native-capability-hook.test.ts`, new workflow-pin test | U5 | +| U10 | Glossary + ADR + doc-targets | `CONTEXT.md`, `docs/adr/0009-*.md`, `docs/agents/doc-targets.yml` | U5 | +| U11 | Claude driver extraction (droppable) | `prove-harness-install.ts`, new shared-types module | U5, U7 | + +### Phase 1 — Release validation (PR 1) + +### U1. Parity ledger and fail-closed coverage + +- **Goal:** Classify every current workflow literal into a ledger entry before any assertion changes, and pin the unparseable-YAML behavior. +- **Requirements:** R9, R10. +- **Dependencies:** none. +- **Files:** `scripts/release-validate.ts` (read), `scripts/release-validate.test.ts`. +- **Approach:** + 1. Enumerate the ~58 whole-file literals (`release-validate.ts:609-668`), the negatives (`:671-681`), the job-scoped checks (`:692-735`), and the action-pin regex (`:600-607`). + 2. Classify each per KTD5's tier flow; resolve the known blocker-shaped entries: `skip-github-release` (comment-satisfied → re-anchor + drop), `ref: ${{ needs.resolve.outputs.candidate_sha }}` (quantifier: name the owning job/step; record the strengthening), `github.event.repository.private == false` (expression-contains), permissions block (deep equality; record dropped adjacency-to-`steps:`). + 3. Add the fail-closed test: corrupt `release.yml` in a repo copy, assert exit 1 and message. +- **Execution note:** Ledger before code — the ledger is the review artifact for R9. The fail-closed test lands red-first against current behavior if current behavior differs. +- **Test scenarios:** + - Unparseable `release.yml` → exit 1, stderr names the parse failure (new coverage). + - Ledger completeness: a test iterates the ledger and asserts every entry names a tier and owner or a drop reason; count equals the enumerated literal count. +- **Verification:** Suite green; ledger reviewed as part of the PR. + +### U2. Structural navigation core + +- **Goal:** Parse `release.yml` once and provide job/step navigation the assertions use. +- **Requirements:** R8, R10. +- **Dependencies:** U1. +- **Files:** `scripts/release-validate.ts`. +- **Approach:** `Bun.YAML.parse` at the current read site (`:507`); helpers to fetch a job, a step by name (or job-scoped run concatenation for unnamed steps), and to walk `jobs.*.steps[].uses`. Job order from `Object.keys`. Fail-closed per KTD6. Keep raw text available for tier-3 checks. +- **Patterns to follow:** `repository-readiness.ts:728-751` (parse + fail-closed shape); `release-validate.test.ts:324` already parses the same file in tests. +- **Test scenarios:** + - Job-order violation (maintain before compatibility in a mutated copy) → exit 1 with the existing "job boundary"-class message. + - Unpinned action ref anywhere in any job → exit 1 (walk replaces the regex). +- **Verification:** Suite green; no assertion behavior changed yet. + +### U3. Migrate assertions to the three tiers + +- **Goal:** Replace the literal loops and string-index job slicing with ledger-backed tiered assertions. +- **Requirements:** R8, R9, R11. +- **Dependencies:** U2. +- **Files:** `scripts/release-validate.ts`. +- **Approach:** Implement each ledger entry in its tier; error messages byte-identical per KTD7; typed comparisons per KTD5 (boolean coercion, expression-contains); tier-3 residuals stay raw-text and scan comments. Delete the slicing code (`:683-691`, `:709-718`) once its assertions are re-homed. +- **Test scenarios (extend existing suite; Test Design Brief first per KTD11):** + - Every existing positive assertion in `release-validate.test.ts:322-533` still passes against the real `release.yml`. + - Every existing mutation test (`:1043-1099`) still produces exit 1 with its asserted stderr substring. + - Reformat-only mutation (reindent a job without semantic change) → validator passes (the brittleness this work removes; new test). + - A required run-fragment moved to a *different* step → exit 1 (step scoping works; new test). +- **Verification:** Suite green; `bun run release:validate` exits 0 on the real repo; ledger has no unimplemented entries. + +### U4. Mutation-anchor guards + +- **Goal:** Make `release-validate.test.ts`'s mutation helpers fail loudly when their anchors stop matching. +- **Requirements:** R11, R12. +- **Dependencies:** U3. +- **Files:** `scripts/release-validate.test.ts`. +- **Approach:** Each string-replace mutation asserts `mutated !== original` before writing (KTD8). +- **Test scenarios:** Covered by the change itself — an anchor that no longer matches turns the test red with a clear message instead of silently passing the unmutated file. +- **Verification:** Suite green; deliberately breaking one anchor locally shows the guard firing. + +### Phase 2 — Harness identity (PR 2) + +### U5. The `harness-identity.ts` module + +- **Goal:** Create the leaf module owning harness identity. +- **Requirements:** R1, R4. +- **Dependencies:** none (Phase 2 start; rebased onto Phase 1). +- **Files:** `scripts/harness-identity.ts`, `scripts/harness-identity.test.ts`. +- **Approach:** Two explicit per-harness records per KTD2 (`hooksDeclarationPath`, `manifestDirectory`, `pluginRootEnvVar`, `displayName`); `type HarnessId` derived from the record keys; separate `QualificationClient` type with client→harness mapping; `codex-desktop` is a value with no other code path (R4). JSDoc with `@example` per `plugin-config.ts` house style; tabs, no semicolons, `as const`. +- **Patterns to follow:** `plugin-config.ts` export shape (interfaces + frozen consts + documented functions); leaf-module rule KTD3. +- **Test scenarios:** + - Record values byte-match the four knowledge points: hooks paths, manifest dirs, env vars (asserting the asymmetry: `PLUGIN_ROOT`, not `CODEX_PLUGIN_ROOT`), display names. + - Client→harness mapping: `claude-cli→claude`, `codex-cli→codex`, `codex-desktop→codex`; exhaustiveness over both types. +- **Verification:** Suite green; module imports nothing from siblings (checked by review; no import lines). + +### U6. Migrate the generators + +- **Goal:** `plugin-config.ts` derives manifests and hook declarations from the module. +- **Requirements:** R2, R3. +- **Dependencies:** U5. +- **Files:** `scripts/plugin-config.ts`. +- **Approach:** `hookDeclarationBody`/`hookDeclaration` (`:443-460`) take `HarnessId` and read env var + path from the records; manifest `hooks:` fields (`:404`, `:436`) derive instead of hardcoding. Generated bytes must not change. +- **Execution note:** Run the deliberate mutation check from KTD11 here: alter a record value, confirm `generate:check` and `native-capability-surface.test.ts` fail, revert. +- **Test scenarios:** + - `generate:check` clean after migration (byte-identical output). + - `native-capability-surface.test.ts` exact-equality assertions unchanged and passing. +- **Verification:** Suite green; `git diff --exit-code -- plugin/` clean. + +### U7. Migrate validators and proofs + +- **Goal:** All remaining lowercase branch sites read the module. +- **Requirements:** R2, R3. +- **Dependencies:** U5, U6; U3 (rebase overlap at `release-validate.ts:526`). +- **Files:** `scripts/build.ts` (`:1766` ternary), `scripts/prove-dx.ts` (`:9`, `:28-31`, `:43-51`), `scripts/release-validate.ts` (`:526-531`), `scripts/prove-harness-install.ts` (unions at `:249`, `:283`, `:1358`; derivations at `:1368-1373`; executables map), `scripts/dev.ts` (delete local `type Harness`, `:59`), `scripts/update.ts` (`:199`), `scripts/prove-distribution.ts` (`:247`, `:253`). +- **Approach:** Site-by-site replacement with module reads; discriminators only, prose messages untouched (Scope Boundaries). The four duplicated hooks-path sites collapse to record reads; the `-cli` journey vocabulary at `prove-harness-install.ts:468/:1556/:1807/:1814` types against `QualificationClient` but keeps its journey-fixture launcher mapping at the call site. +- **Execution note:** Deliberate mutation check on the four hooks-path assertion sites (KTD11) before trusting the suite. +- **Test scenarios:** + - Existing per-file tests pass unchanged (message parity per KTD7). + - Grep-style orphan check: no remaining `"claude" | "codex"` union declarations outside `harness-identity.ts` (discriminators only). +- **Verification:** Suite green after each file's migration (commit-sized steps). + +### U8. Display vocabulary migration + +- **Goal:** Delete the capitalized vocabulary as a standalone type; derive display names from the module. +- **Requirements:** R2, R3. +- **Dependencies:** U5. +- **Files:** `scripts/harness-install-recovery.ts` (`:61-95`), `scripts/prove-harness-install.ts` (`:916`), `scripts/harness-install-codex.ts` (`:122`), `scripts/codex-production-update.ts` (`harness: "codex"` field sites `:191`, `:1267`, `:1386`, `:1462`). +- **Approach:** `HarnessRecoveryAdapter.harness` becomes `HarnessId`; interpolated messages use `displayName` — every thrown message byte-identical (`harness-install-recovery.test.ts:78` asserts them). `codex-production-update`'s lowercase `harness` field types against `HarnessId` (resolves the two-casings collision hazard with its import from `harness-install-recovery`). +- **Test scenarios:** + - `harness-install-recovery.test.ts` passes unchanged — asserted messages still contain "Claude"/"Codex" capitalized. + - Codex production-update envelope JSON unchanged (`harness: "codex"` persists lowercase). +- **Verification:** Suite green. + +### U9. Non-TypeScript parity tests + +- **Goal:** The shell hook and workflow pin line cannot silently drift from the module. +- **Requirements:** R5. +- **Dependencies:** U5. +- **Files:** `scripts/native-capability-hook.test.ts` (extend), new `scripts/workflow-pin-parity.test.ts` (or sibling location per Test Design Brief). +- **Approach:** Hook test reads `plugin/hooks/native-capability-hook` and asserts its `SessionStart:`/`Stop:` case arms cover exactly the module's harness IDs. Pin-line test extracts the `bun add --global` line from `plugin-ci.yml`, `hosted-canary.yml`, `release.yml` and asserts all three are byte-identical — version-agnostic per KTD4. +- **Test scenarios:** + - Case-arm drift (add/remove/rename an ID in a repo copy) → test fails. + - One workflow's pin line differing from the others → test fails; bumping all three together → passes. +- **Verification:** Suite green. + +### U10. Glossary, ADR, doc-targets + +- **Goal:** The vocabulary decision and the new term are durable and drift-guarded. +- **Requirements:** R6. +- **Dependencies:** U5 (definition crystallised from the real module). +- **Files:** `CONTEXT.md`, `docs/adr/0009-canonical-harness-identity.md`, `docs/agents/doc-targets.yml`. +- **Approach:** Glossary entry in house format (`**Term**:` + definition + `_Avoid_:` line) naming the module's term; ADR 0009 in ADR-0008's sectioned form (Status/Context/Decision) records: canonical lowercase IDs, the frozen `-cli` qualification-client contract with `codex-desktop`, the display mapping, and the driver-seam constraint stated natively (KTD9) with cross-references to ADR-0001 and ADR-0003. doc-targets.yml rows bind both to `scripts/harness-identity.ts`. +- **Test scenarios:** Test expectation: none — documentation unit; drift coverage comes from the doc-targets rows and existing docs-drift tooling. +- **Verification:** `doc-targets.yml` parses; docs-drift check (if run) passes. + +### U11. Claude driver extraction (droppable) + +- **Goal:** Claude's inlined driver (`prove-harness-install.ts:845-1016`) sits behind a Claude-specific DI interface. +- **Requirements:** R7, R3. +- **Dependencies:** U5, U7. +- **Files:** `scripts/prove-harness-install.ts`, `scripts/harness-install-claude.ts` (new), shared install-state types module if needed to avoid deepening the existing type-only cycle with `harness-install-codex.ts`. +- **Approach:** Per KTD9 — same DI pattern as `CodexDriverDependencies`, Claude-shaped surface (three-scope loop; `findClaudeInstall`/`replaceClaudeInstall`/`claudeEnvironment` also serve `proveHostedHarnessInstall:1175-1198`, so extraction must serve both callers). The 12 named exports `prove-harness-install.test.ts` imports remain stable. Scope preservation semantics (user/project/local, `--keep-data`, `defaultEnabled: false`) unchanged. +- **Execution note:** This unit is droppable to follow-up at implementation time without unwinding U5–U10. If dropped, record it under Deferred to Follow-Up Work in the PR description. +- **Test scenarios:** + - `prove-harness-install.test.ts` passes unchanged (export surface stable). + - Injected-failure recovery paths for Claude behave identically (existing recovery tests). +- **Verification:** Suite green; `bun run prove:harness-install` unchanged behavior locally where runnable. + +--- + +## Verification Contract + +| Gate | Command / mechanism | Applies to | +|---|---|---| +| Unit + integration suite | Bun tests via the repo's MCP runner (never raw `bun test`) | every unit (R12) | +| Generated-output drift | `bun run generate:check`; `git diff --exit-code -- plugin/` | U5–U8 | +| Release validation self-check | `bun run release:validate` exits 0 on the real repo | U1–U4, U7 | +| Full proof chain | `bun run prove:all` before each PR | both PRs | +| Deliberate mutation checks | KTD11: break a derived value, observe a failure, revert | U6, U7 | +| Test Design Brief | `test-design` skill before any test-artifact change | U1, U3, U4, U5, U9 | +| Docs drift | doc-targets.yml rows verified by the docs-drift tooling | U10 | + +## Definition of Done + +- All units complete, or U11 consciously dropped and recorded as deferred. +- Parity ledger covers every enumerated literal with a tier or recorded drop; no unclassified entries (R9). +- Suite green, `generate:check` clean, `release:validate` exit 0, `prove:all` green at each PR head. +- Generated `plugin/` output, persisted JSON contracts, and asserted error messages byte-identical throughout (R3, R4, R11). +- Glossary entry, ADR 0009, and doc-targets rows landed (R6). +- No abandoned or experimental code in either diff; both PRs opened from the myagentdojo account with hosted-canary qualification. diff --git a/scripts/release-validate.test.ts b/scripts/release-validate.test.ts index 9c38484..6abe1d6 100644 --- a/scripts/release-validate.test.ts +++ b/scripts/release-validate.test.ts @@ -21,6 +21,7 @@ import { validateRepairBinding, validateResumeCandidateBinding, } from "./release-validate" +import { RELEASE_WORKFLOW_PARITY_LEDGER } from "./release-workflow-parity" const root = resolve(import.meta.dir, "..") const ignoredEntries = new Set([".dev", ".git", ".worktrees", "dist", "node_modules"]) @@ -64,6 +65,337 @@ function validateWithArguments(cwd: string, arguments_: string[]): ReturnType { + // Current release-validate.ts provenance: 1 action-pin + 67 whole-file + 4 negative/top-level + // + 4 job-boundary + 8 maintain-required + 1 maintain-forbidden + 3 release-job = 88. + const enumeratedLiteralCount = 88 + + expect(RELEASE_WORKFLOW_PARITY_LEDGER).toHaveLength(enumeratedLiteralCount) + for (const entry of RELEASE_WORKFLOW_PARITY_LEDGER) { + expect(entry.literal.length).toBeGreaterThan(0) + if ("dropReason" in entry) { + expect(entry.dropReason.length).toBeGreaterThan(0) + continue + } + expect(["structural", "step-run", "raw-residual"]).toContain(entry.tier) + expect(entry.owner.length).toBeGreaterThan(0) + expect(entry.comparison.length).toBeGreaterThan(0) + } +}) + +test("release validation fails closed when the release workflow is not parseable YAML", () => { + const temporaryRoot = copyRepository() + try { + writeFileSync(join(temporaryRoot, ".github", "workflows", "release.yml"), "jobs: [\n") + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain("release workflow YAML could not be parsed") + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation accepts semantic-preserving workflow reformatting", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " group: release-maintenance\n", + " group : release-maintenance\n", + ) + expect(mutated !== original, "semantic reformatting mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode, result.stderr.toString()).toBe(0) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects an unpinned job-level reusable workflow", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + "\n converge:\n", + "\n reusable-call:\n uses: myagentdojo/agent-plugin-template/.github/workflows/release.yml@main\n\n converge:\n", + ) + expect(mutated !== original, "job-level uses mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow actions must be pinned to full commit SHAs", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation accepts a repository-local action reference", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6\n", + " - uses: ./.github/actions/local-setup\n - uses: oven-sh/setup-bun@0c5077e51419868618aeaa5fe8019c62421857d6\n", + ) + expect(mutated !== original, "local action reference mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode, result.stderr.toString()).toBe(0) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects a second checkout step that skips the pinned ref", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1\n with:\n ref: ${{ needs.resolve.outputs.candidate_sha }}\n", + " - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1\n with:\n ref: ${{ needs.resolve.outputs.candidate_sha }}\n - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1\n with:\n ref: main\n persist-credentials: false\n", + ) + expect(mutated !== original, "second checkout step mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow is missing ref: ${{ needs.resolve.outputs.candidate_sha }}", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation scopes required run fragments to their owning step", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const removed = original.replace( + " run: |\n bun run prove:all\n", + " run: \"true\"\n", + ) + expect(removed !== original, "prove:all removal mutation anchor must match").toBe(true) + const workflow = removed.replace( + " - name: Reject generated release-surface drift\n run: git diff --exit-code -- plugin/\n", + " - name: Reject generated release-surface drift\n run: |\n git diff --exit-code -- plugin/\n bun run prove:all\n", + ) + expect(workflow !== removed, "prove:all relocation mutation anchor must match").toBe(true) + writeFileSync(workflowPath, workflow) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain("release workflow is missing bun run prove:all") + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation scopes the publish concurrency group to the release job", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const removed = original.replace( + " concurrency:\n group: release-publication-${{ needs.resolve.outputs.release_tag }}\n cancel-in-progress: false\n runs-on: ubuntu-24.04\n environment: release\n", + " runs-on: ubuntu-24.04\n environment: release\n", + ) + expect(removed !== original, "publish concurrency removal mutation anchor must match").toBe(true) + const mutated = removed.replace( + " converge:\n name: Require release operation convergence\n if: always()\n", + " converge:\n name: Require release operation convergence\n if: always()\n concurrency:\n group: release-publication-${{ needs.resolve.outputs.release_tag }}\n cancel-in-progress: false\n", + ) + expect(mutated !== removed, "publish concurrency relocation mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow publish mutation must serialize by resolved immutable tag", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects a GITHUB_TOKEN fallback nested in the maintain job", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " - name: Detect a merged release candidate stranded before its tag\n env:\n GH_TOKEN: ${{ github.token }}\n", + " - name: Detect a merged release candidate stranded before its tag\n env:\n GH_TOKEN: ${{ github.token }}\n FALLBACK_TOKEN: ${{ secrets.GITHUB_TOKEN }}\n", + ) + expect(mutated !== original, "maintain GITHUB_TOKEN mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow maintenance job must not fall back to GITHUB_TOKEN", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects a drifted resolve-step PUSH_BEFORE_SHA expression", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + "PUSH_BEFORE_SHA: ${{ github.event.before }}", + "PUSH_BEFORE_SHA: ${{ github.event.after }}", + ) + expect(mutated !== original, "PUSH_BEFORE_SHA mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow is missing PUSH_BEFORE_SHA: ${{ github.event.before }}", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects a broken release-candidate artifact producer/consumer pair", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " name: release-candidate-${{ github.run_id }}\n", + " name: renamed-release-candidate-${{ github.run_id }}\n", + ) + expect(mutated !== original, "release-candidate artifact mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow is missing release-candidate-${{ github.run_id }}", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects a compatibility matrix missing a required runner", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " runner: macos-15-intel\n", + " runner: macos-15-intel-renamed\n", + ) + expect(mutated !== original, "matrix runner mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain("release workflow is missing macos-15-intel") + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects a release job that no longer depends on package", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " needs:\n - resolve\n - package\n", + " needs:\n - resolve\n", + ) + expect(mutated !== original, "release needs mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow publish job must depend on package", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects an attestation step without the public-repository condition", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " if: github.event.repository.private == false && steps.attestation.outputs.needed == 'true'\n", + " if: steps.attestation.outputs.needed == 'true'\n", + ) + expect(mutated !== original, "attestation condition mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow is missing github.event.repository.private == false", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + +test("release validation rejects a checkout ref outside the resolved candidate", () => { + const temporaryRoot = copyRepository() + try { + const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " ref: ${{ needs.resolve.outputs.candidate_sha }}\n", + " ref: ${{ github.sha }}\n", + ) + expect(mutated !== original, "checkout ref mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) + + const result = validate(temporaryRoot) + + expect(result.exitCode).toBe(1) + expect(result.stderr.toString()).toContain( + "release workflow is missing ref: ${{ needs.resolve.outputs.candidate_sha }}", + ) + } finally { + rmSync(temporaryRoot, { recursive: true, force: true }) + } +}) + const allowedProjection = [ ".claude-plugin/marketplace.json", ".github/.release-please-manifest.json", @@ -1162,10 +1494,10 @@ test.each([ const temporaryRoot = copyRepository() try { const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") - writeFileSync( - workflowPath, - readFileSync(workflowPath, "utf8").replace(marker, "\n renamed-job:\n"), - ) + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace(marker, "\n renamed-job:\n") + expect(mutated !== original, "job boundary mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) const result = validate(temporaryRoot) @@ -1179,13 +1511,13 @@ test.each([ test("release validation rejects read-only pull-request lineage permissions", () => { const temporaryRoot = copyRepository() const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") - writeFileSync( - workflowPath, - readFileSync(workflowPath, "utf8").replace( - " issues: write\n pull-requests: write\n steps:\n", - " issues: write\n pull-requests: read\n steps:\n", - ), + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " issues: write\n pull-requests: write\n steps:\n", + " issues: write\n pull-requests: read\n steps:\n", ) + expect(mutated !== original, "pull-request permission mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) const result = validate(temporaryRoot) @@ -1196,13 +1528,13 @@ test("release validation rejects read-only pull-request lineage permissions", () test("release validation rejects extra protected release permissions", () => { const temporaryRoot = copyRepository() const workflowPath = join(temporaryRoot, ".github", "workflows", "release.yml") - writeFileSync( - workflowPath, - readFileSync(workflowPath, "utf8").replace( - " pull-requests: write\n steps:\n", - " pull-requests: write\n packages: write\n steps:\n", - ), + const original = readFileSync(workflowPath, "utf8") + const mutated = original.replace( + " pull-requests: write\n steps:\n", + " pull-requests: write\n packages: write\n steps:\n", ) + expect(mutated !== original, "extra permission mutation anchor must match").toBe(true) + writeFileSync(workflowPath, mutated) const result = validate(temporaryRoot) diff --git a/scripts/release-validate.ts b/scripts/release-validate.ts index 72f9e15..c0f61eb 100644 --- a/scripts/release-validate.ts +++ b/scripts/release-validate.ts @@ -5,6 +5,7 @@ import { validateBunOnlyPayload } from "./build" import { checkNativeCapabilityFixture } from "./native-capability-fixture" import { checkGeneratedFiles, loadPluginConfig } from "./plugin-config" import { RELEASE_PROJECTION_PATH_SET } from "./release-projection" +import { validateReleaseWorkflowParity } from "./release-workflow-parity" const root = resolve(import.meta.dir, "..") @@ -561,6 +562,12 @@ function validateRepository(repositoryRoot: string) { const releaseManifest = readJson(repositoryRoot, ".github/.release-please-manifest.json") const releaseConfig = readJson(repositoryRoot, ".github/release-please-config.json") const releaseWorkflow = readFileSync(join(repositoryRoot, ".github/workflows/release.yml"), "utf8") + let parsedReleaseWorkflow: unknown + try { + parsedReleaseWorkflow = Bun.YAML.parse(releaseWorkflow) + } catch { + throw new Error("release workflow YAML could not be parsed") + } const changelog = readFileSync(join(repositoryRoot, "CHANGELOG.md"), "utf8") const version = pluginConfig.version @@ -653,152 +660,7 @@ function validateRepository(repositoryRoot: string) { } } - const actionReferences = [...releaseWorkflow.matchAll(/uses: [^@\s]+@([^\s]+)/g)].map( - (match) => match[1], - ) - if ( - actionReferences.length === 0 || - actionReferences.some((reference) => !/^[a-f0-9]{40}$/.test(reference)) - ) { - throw new Error("release workflow actions must be pinned to full commit SHAs") - } - for (const required of [ - "skip-github-release", - "publication-candidate-${GITHUB_SHA}", - "merge_commit_sha", - "EXPECTED_RELEASE_PLEASE_LOGIN", - "PUSH_BEFORE_SHA: ${{ github.event.before }}", - "PUSH_FORCED: ${{ github.event.forced }}", - 'if [[ "$GITHUB_REF" != "refs/heads/${BASE_BRANCH}" ]]', - 'if [[ "$PUSH_FORCED" != "false" ]]', - 'git merge-base --is-ancestor "$PUSH_BEFORE_SHA" "$GITHUB_SHA"', - "candidate_parent_shas", - "merged_pr_base_sha", - "reviewed_pr_head_sha", - "trusted_base_sha", - "admitPublicationCandidate", - "validateResumeCandidateBinding", - 'if [[ "$OPERATION" == "resume" ]]', - "gh api --include", - "404) return 0 ;;", - "Could not prove whether tag", - "publication-candidate-${RESUME_SHA}", - "Resume requires the persisted publication candidate", - "Detect a merged release candidate stranded before its tag", - "-f operation=resume", - "scripts/release-projection.ts", - "bun run prove:all", - "git diff --exit-code -- plugin/", - "ubuntu-24.04-arm", - "macos-15-intel", - "SOURCE_COMMIT", - "ref: ${{ needs.resolve.outputs.candidate_sha }}", - "workflow_policy_sha=$(git rev-parse HEAD)", - 'git checkout --detach "$workflow_policy_sha"', - 'trusted_base_sha=$(git rev-parse "${candidate_sha}^1")', - 'git checkout --detach "$trusted_base_sha"', - 'TRUSTED_BASE_SHA: ${{ needs.resolve.outputs.trusted_base_sha }}', - 'ADMITTED_MERGED_PR_BASE_SHA: ${{ needs.resolve.outputs.merged_pr_base_sha }}', - 'ADMITTED_REVIEWED_PR_HEAD_SHA: ${{ needs.resolve.outputs.reviewed_pr_head_sha }}', - 'trusted_base_sha="$TRUSTED_BASE_SHA"', - 'if [[ "$merged_pr_base_sha" != "$ADMITTED_MERGED_PR_BASE_SHA" || "$reviewed_pr_head_sha" != "$ADMITTED_REVIEWED_PR_HEAD_SHA" ]]', - 'git checkout --detach "$CANDIDATE_SHA"', - 'if [[ "$(git rev-parse HEAD)" != "$CANDIDATE_SHA" ]]', - "tag -a \"$RELEASE_TAG\" \"$CANDIDATE_SHA\" -F persisted-candidate.json", - "git for-each-ref --format='%(contents)'", - 'gh api "repos/${GITHUB_REPOSITORY}/pulls/${pr_number}"', - "trusted-repair-candidate.json", - "validateRepairCandidateBinding", - "git push origin \"refs/tags/${RELEASE_TAG}\"", - "remote_tag_sha", - "gh release create", - "--verify-tag", - "gh release download", - "gh release upload", - "*.checksums.json", - "replace_mismatched_assets", - "sha256sum", - "group: release-maintenance", - "group: release-publication-${{ needs.resolve.outputs.release_tag }}", - "release-candidate-${{ github.run_id }}", - "release-platform-candidate-${{ github.run_id }}", - "bun run prove:runtime-platform", - "--fixture-acknowledged", - 'cmp --silent "$candidate_archive" "$rebuilt_archive"', - "overwrite: true", - "environment: release", - "gh attestation verify", - "actions/attest", - "github.event.repository.private == false", - ]) { - if (!releaseWorkflow.includes(required)) throw new Error(`release workflow is missing ${required}`) - } - for (const forbidden of ["parent_count", "mergeMode"]) { - if (releaseWorkflow.includes(forbidden)) { - throw new Error(`release workflow retains unsupported merge-shape metadata: ${forbidden}`) - } - } - if (releaseWorkflow.includes("github.run_attempt")) { - throw new Error("release workflow artifact identity must survive rerun-failed-jobs attempts") - } - if (/^concurrency:/m.test(releaseWorkflow)) { - throw new Error("release workflow must serialize only mutation jobs, not discard distinct pending runs") - } - const maintainJobStart = releaseWorkflow.indexOf("\n maintain:\n") - const compatibilityJobStart = releaseWorkflow.indexOf("\n compatibility:\n") - if ( - maintainJobStart === -1 || - compatibilityJobStart === -1 || - compatibilityJobStart <= maintainJobStart - ) { - throw new Error("release workflow is missing the maintain or compatibility job boundary") - } - const maintainJob = releaseWorkflow.slice(maintainJobStart, compatibilityJobStart) - for (const required of [ - "group: release-maintenance", - "cancel-in-progress: false", - "persist-credentials: false", - "id: bootstrap-version", - "jq 'length' .github/.release-please-manifest.json", - 'release_as="0.1.0"', - "token: ${{ secrets.RELEASE_PLEASE_TOKEN }}", - "release-as: ${{ steps.bootstrap-version.outputs.release_as }}", - ]) { - if (!maintainJob.includes(required)) { - throw new Error(`release workflow maintenance job is missing ${required}`) - } - } - if (maintainJob.includes("secrets.GITHUB_TOKEN")) { - throw new Error("release workflow maintenance job must not fall back to GITHUB_TOKEN") - } - const releaseJobStart = releaseWorkflow.indexOf("\n release:\n") - if (releaseJobStart === -1) throw new Error("release workflow is missing the release job boundary") - const convergeJobStart = releaseWorkflow.indexOf("\n converge:\n") - if (convergeJobStart === -1) { - throw new Error("release workflow is missing the converge job boundary") - } - if (convergeJobStart <= releaseJobStart) { - throw new Error("release workflow converge job must follow the release job") - } - const releaseJob = releaseWorkflow.slice(releaseJobStart, convergeJobStart) - if (!releaseJob.includes(" needs:\n - resolve\n - package\n")) { - throw new Error("release workflow publish job must depend on package") - } - if (!releaseJob.includes("group: release-publication-${{ needs.resolve.outputs.release_tag }}")) { - throw new Error("release workflow publish mutation must serialize by resolved immutable tag") - } - const requiredReleasePermissionsBlock = - " permissions:\n" + - " actions: read\n" + - " contents: write\n" + - " id-token: write\n" + - " attestations: write\n" + - " issues: write\n" + - " pull-requests: write\n" + - " steps:\n" - if (!releaseJob.includes(requiredReleasePermissionsBlock)) { - throw new Error("release workflow publish job permissions must match the protected release contract") - } + validateReleaseWorkflowParity(releaseWorkflow, parsedReleaseWorkflow) return { ok: true, diff --git a/scripts/release-workflow-parity.ts b/scripts/release-workflow-parity.ts new file mode 100644 index 0000000..06a87d5 --- /dev/null +++ b/scripts/release-workflow-parity.ts @@ -0,0 +1,1027 @@ +type ReleaseWorkflowStep = Record +type ReleaseWorkflowJob = Record + +function isRecord(value: unknown): value is Record { + return typeof value === "object" && value !== null && !Array.isArray(value) +} + +/** Return parsed jobs in their YAML insertion order. */ +function releaseWorkflowJobs(workflow: unknown): Record { + if (!isRecord(workflow) || !isRecord(workflow.jobs)) return {} + return Object.fromEntries( + Object.entries(workflow.jobs).filter( + (entry): entry is [string, ReleaseWorkflowJob] => isRecord(entry[1]), + ), + ) +} + +/** Fetch one parsed workflow job without assuming the YAML document shape. */ +function releaseWorkflowJob(workflow: unknown, jobName: string): ReleaseWorkflowJob | undefined { + return releaseWorkflowJobs(workflow)[jobName] +} + +/** Return the object-shaped steps from one parsed workflow job. */ +function releaseWorkflowSteps(job: ReleaseWorkflowJob | undefined): ReleaseWorkflowStep[] { + if (!Array.isArray(job?.steps)) return [] + return job.steps.filter((step): step is ReleaseWorkflowStep => isRecord(step)) +} + +/** Fetch one named step from a parsed workflow job. */ +function releaseWorkflowStep( + job: ReleaseWorkflowJob | undefined, + stepName: string, +): ReleaseWorkflowStep | undefined { + return releaseWorkflowSteps(job).find((step) => step.name === stepName) +} + +/** Walk every parsed job and job step and preserve each uses value for pin validation. */ +function releaseWorkflowActionReferences(workflow: unknown): unknown[] { + return Object.values(releaseWorkflowJobs(workflow)).flatMap((job) => [ + ...(Object.hasOwn(job, "uses") ? [job.uses] : []), + ...releaseWorkflowSteps(job) + .filter((step) => Object.hasOwn(step, "uses")) + .map((step) => step.uses), + ]) +} + +/** + * Repository-local actions ship with the checked-out revision and carry no ref to pin. + * Requiring a commit SHA here would reject `uses: ./.github/actions/` outright. + */ +function releaseWorkflowLocalActionReference(reference: unknown): boolean { + return typeof reference === "string" && reference.startsWith("./") +} + +export type ReleaseWorkflowParityTier = "structural" | "step-run" | "raw-residual" + +export type ReleaseWorkflowParityLedgerEntry = + | { + literal: string + tier: ReleaseWorkflowParityTier + owner: string + comparison: string + note?: string + droppedAspect?: string + failureMessage?: string + } + | { + literal: string + dropReason: string + } + +/** Migration ledger for every release-workflow assertion in the current raw validator. */ +export const RELEASE_WORKFLOW_PARITY_LEDGER = [ + { + literal: "uses: @ + full-commit-SHA ref", + tier: "structural", + owner: "jobs.*.uses + jobs.*.steps[].uses", + comparison: "all action references match a 40-character lowercase commit SHA", + }, + { + literal: "skip-github-release", + dropReason: + "The workflow occurrence is comment-only; .github/release-please-config.json skip-github-release=true remains the executable assertion owner.", + }, + { + literal: "publication-candidate-${GITHUB_SHA}", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "merge_commit_sha", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "EXPECTED_RELEASE_PLEASE_LOGIN", + tier: "structural", + owner: + "jobs.resolve step Resolve unique candidate or immutable repair tag env.EXPECTED_RELEASE_PLEASE_LOGIN", + comparison: "field present", + }, + { + literal: "PUSH_BEFORE_SHA: ${{ github.event.before }}", + tier: "structural", + owner: + "jobs.resolve step Resolve unique candidate or immutable repair tag env.PUSH_BEFORE_SHA", + comparison: "equals ${{ github.event.before }}", + }, + { + literal: "PUSH_FORCED: ${{ github.event.forced }}", + tier: "structural", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag env.PUSH_FORCED", + comparison: "equals ${{ github.event.forced }}", + }, + { + literal: 'if [[ "$GITHUB_REF" != "refs/heads/${BASE_BRANCH}" ]]', + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: 'if [[ "$PUSH_FORCED" != "false" ]]', + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: 'git merge-base --is-ancestor "$PUSH_BEFORE_SHA" "$GITHUB_SHA"', + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "candidate_parent_shas", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "merged_pr_base_sha", + tier: "structural", + owner: "jobs.resolve.outputs.merged_pr_base_sha", + comparison: "field present", + }, + { + literal: "reviewed_pr_head_sha", + tier: "structural", + owner: "jobs.resolve.outputs.reviewed_pr_head_sha", + comparison: "field present", + }, + { + literal: "trusted_base_sha", + tier: "structural", + owner: "jobs.resolve.outputs.trusted_base_sha", + comparison: "field present", + }, + { + literal: "admitPublicationCandidate", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "validateResumeCandidateBinding", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: 'if [[ "$OPERATION" == "resume" ]]', + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "gh api --include", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "404) return 0 ;;", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "Could not prove whether tag", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "publication-candidate-${RESUME_SHA}", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "Resume requires the persisted publication candidate", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "Detect a merged release candidate stranded before its tag", + tier: "structural", + owner: "jobs.maintain.steps[].name", + comparison: "contains exact step name", + }, + { + literal: "-f operation=resume", + tier: "step-run", + owner: "jobs.maintain step Detect a merged release candidate stranded before its tag run", + comparison: "contains", + }, + { + literal: "scripts/release-projection.ts", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "bun run prove:all", + tier: "step-run", + owner: "jobs.package step Validate and prove release payload run", + comparison: "contains", + }, + { + literal: "git diff --exit-code -- plugin/", + tier: "step-run", + owner: "jobs.package step Reject generated release-surface drift run", + comparison: "contains", + }, + { + literal: "ubuntu-24.04-arm", + tier: "structural", + owner: "jobs.compatibility.strategy.matrix.include[].runner", + comparison: "array contains", + }, + { + literal: "macos-15-intel", + tier: "structural", + owner: "jobs.compatibility.strategy.matrix.include[].runner", + comparison: "array contains", + }, + { + literal: "SOURCE_COMMIT", + tier: "structural", + owner: "jobs.package step Validate and prove release payload env.SOURCE_COMMIT", + comparison: "field present", + }, + { + literal: "ref: ${{ needs.resolve.outputs.candidate_sha }}", + tier: "structural", + owner: "jobs.{candidate,compatibility,package,release} checkout steps with.ref", + comparison: "every owner equals ${{ needs.resolve.outputs.candidate_sha }}", + note: "Strengthens the raw existential check by naming all four candidate checkout owners.", + }, + { + literal: "workflow_policy_sha=$(git rev-parse HEAD)", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: 'git checkout --detach "$workflow_policy_sha"', + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: 'trusted_base_sha=$(git rev-parse "${candidate_sha}^1")', + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: 'git checkout --detach "$trusted_base_sha"', + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "TRUSTED_BASE_SHA: ${{ needs.resolve.outputs.trusted_base_sha }}", + tier: "structural", + owner: "jobs.release step Replay current publication admission before mutation env.TRUSTED_BASE_SHA", + comparison: "equals ${{ needs.resolve.outputs.trusted_base_sha }}", + }, + { + literal: "ADMITTED_MERGED_PR_BASE_SHA: ${{ needs.resolve.outputs.merged_pr_base_sha }}", + tier: "structural", + owner: + "jobs.release step Replay current publication admission before mutation env.ADMITTED_MERGED_PR_BASE_SHA", + comparison: "equals ${{ needs.resolve.outputs.merged_pr_base_sha }}", + }, + { + literal: "ADMITTED_REVIEWED_PR_HEAD_SHA: ${{ needs.resolve.outputs.reviewed_pr_head_sha }}", + tier: "structural", + owner: + "jobs.release step Replay current publication admission before mutation env.ADMITTED_REVIEWED_PR_HEAD_SHA", + comparison: "equals ${{ needs.resolve.outputs.reviewed_pr_head_sha }}", + }, + { + literal: 'trusted_base_sha="$TRUSTED_BASE_SHA"', + tier: "step-run", + owner: "jobs.release step Replay current publication admission before mutation run", + comparison: "contains", + }, + { + literal: + 'if [[ "$merged_pr_base_sha" != "$ADMITTED_MERGED_PR_BASE_SHA" || "$reviewed_pr_head_sha" != "$ADMITTED_REVIEWED_PR_HEAD_SHA" ]]', + tier: "step-run", + owner: "jobs.release step Replay current publication admission before mutation run", + comparison: "contains", + }, + { + literal: 'git checkout --detach "$CANDIDATE_SHA"', + tier: "step-run", + owner: "jobs.release step Replay current publication admission before mutation run", + comparison: "contains", + }, + { + literal: 'if [[ "$(git rev-parse HEAD)" != "$CANDIDATE_SHA" ]]', + tier: "step-run", + owner: "jobs.release step Replay current publication admission before mutation run", + comparison: "contains", + }, + { + literal: 'tag -a "$RELEASE_TAG" "$CANDIDATE_SHA" -F persisted-candidate.json', + tier: "step-run", + owner: "jobs.release step Create or verify immutable tag run", + comparison: "contains", + }, + { + literal: "git for-each-ref --format='%(contents)'", + tier: "step-run", + owner: "jobs.release step Create or verify immutable tag run", + comparison: "contains", + }, + { + literal: 'gh api "repos/${GITHUB_REPOSITORY}/pulls/${pr_number}"', + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "trusted-repair-candidate.json", + tier: "step-run", + owner: "jobs.resolve step Resolve unique candidate or immutable repair tag run", + comparison: "contains", + }, + { + literal: "validateRepairCandidateBinding", + tier: "step-run", + owner: "jobs.release step Replay current publication admission before mutation run", + comparison: "contains", + }, + { + literal: 'git push origin "refs/tags/${RELEASE_TAG}"', + tier: "step-run", + owner: "jobs.release step Create or verify immutable tag run", + comparison: "contains", + }, + { + literal: "remote_tag_sha", + tier: "step-run", + owner: "jobs.release step Create or verify immutable tag run", + comparison: "contains", + }, + { + literal: "gh release create", + tier: "step-run", + owner: "jobs.release step Create missing GitHub Release and validate target run", + comparison: "contains", + }, + { + literal: "--verify-tag", + tier: "step-run", + owner: "jobs.release step Create missing GitHub Release and validate target run", + comparison: "contains", + }, + { + literal: "gh release download", + tier: "step-run", + owner: "jobs.release step Compare release assets before mutation run", + comparison: "contains", + }, + { + literal: "gh release upload", + tier: "step-run", + owner: "jobs.release step Add or replace admitted release assets run", + comparison: "contains", + }, + { + literal: "*.checksums.json", + tier: "structural", + owner: "jobs.package upload-artifact step with.path", + comparison: "parsed block scalar contains", + }, + { + literal: "replace_mismatched_assets", + tier: "structural", + owner: "on.workflow_dispatch.inputs.replace_mismatched_assets", + comparison: "field present", + }, + { + literal: "sha256sum", + tier: "step-run", + owner: "jobs.release step Compare release assets before mutation run", + comparison: "contains", + }, + { + literal: "group: release-maintenance", + tier: "structural", + owner: "workflow (anywhere)", + comparison: "group value scalar present anywhere in the parsed workflow", + }, + { + literal: "group: release-publication-${{ needs.resolve.outputs.release_tag }}", + tier: "structural", + owner: "workflow (anywhere)", + comparison: "group value scalar present anywhere in the parsed workflow", + }, + { + literal: "release-candidate-${{ github.run_id }}", + tier: "structural", + owner: "jobs.package upload-artifact with.name and jobs.release download env.ARTIFACT_NAME", + comparison: "producer and consumer equal", + }, + { + literal: "release-platform-candidate-${{ github.run_id }}", + tier: "structural", + owner: + "jobs.candidate upload-artifact with.name and jobs.{compatibility,package} download env.ARTIFACT_NAME", + comparison: "producer and consumers equal", + }, + { + literal: "bun run prove:runtime-platform", + tier: "step-run", + owner: "jobs.compatibility step Prove packaged runtime custody on this target run", + comparison: "contains", + }, + { + literal: "--fixture-acknowledged", + tier: "step-run", + owner: "jobs.compatibility step Prove packaged runtime custody on this target run", + comparison: "contains", + }, + { + literal: 'cmp --silent "$candidate_archive" "$rebuilt_archive"', + tier: "step-run", + owner: "jobs.package step Compare the rebuilt package with the platform-proven candidate run", + comparison: "contains", + }, + { + literal: "overwrite: true", + tier: "structural", + owner: "jobs.{candidate,package} upload-artifact steps with.overwrite", + comparison: "every owner equals boolean true", + }, + { + literal: "environment: release", + tier: "structural", + owner: "jobs.release.environment", + comparison: "equals release", + }, + { + literal: "gh attestation verify", + tier: "step-run", + owner: "jobs.release step Check for existing matching public attestation run", + comparison: "contains", + }, + { + literal: "actions/attest", + tier: "structural", + owner: "jobs.release step Add missing public release attestation uses", + comparison: "action name equals actions/attest", + }, + { + literal: "github.event.repository.private == false", + tier: "structural", + owner: + "jobs.release steps Check for existing matching public attestation and Add missing public release attestation if", + comparison: "every owner expression contains", + }, + { + literal: "parent_count", + tier: "raw-residual", + owner: ".github/workflows/release.yml raw text", + comparison: "forbidden anywhere including comments", + }, + { + literal: "mergeMode", + tier: "raw-residual", + owner: ".github/workflows/release.yml raw text", + comparison: "forbidden anywhere including comments", + }, + { + literal: "github.run_attempt", + tier: "raw-residual", + owner: ".github/workflows/release.yml raw text", + comparison: "forbidden anywhere including comments", + }, + { + literal: "/^concurrency:/m", + tier: "structural", + owner: "workflow.concurrency", + comparison: "field absent", + }, + { + literal: "\n maintain:\n", + tier: "structural", + owner: "jobs.maintain", + comparison: "field present before jobs.compatibility", + }, + { + literal: "\n compatibility:\n", + tier: "structural", + owner: "jobs.compatibility", + comparison: "field present after jobs.maintain", + }, + { + literal: "\n release:\n", + tier: "structural", + owner: "jobs.release", + comparison: "field present before jobs.converge", + }, + { + literal: "\n converge:\n", + tier: "structural", + owner: "jobs.converge", + comparison: "field present after jobs.release", + }, + { + literal: "group: release-maintenance", + tier: "structural", + owner: "jobs.maintain.concurrency.group", + comparison: "equals release-maintenance", + failureMessage: "release workflow maintenance job is missing group: release-maintenance", + }, + { + literal: "cancel-in-progress: false", + tier: "structural", + owner: "jobs.maintain.concurrency.cancel-in-progress", + comparison: "equals boolean false", + }, + { + literal: "persist-credentials: false", + tier: "structural", + owner: "jobs.maintain checkout step with.persist-credentials", + comparison: "equals boolean false", + }, + { + literal: "id: bootstrap-version", + tier: "structural", + owner: "jobs.maintain step Pin only the first release to v0.1.0 id", + comparison: "equals bootstrap-version", + }, + { + literal: "jq 'length' .github/.release-please-manifest.json", + tier: "step-run", + owner: "jobs.maintain step Pin only the first release to v0.1.0 run", + comparison: "contains", + }, + { + literal: 'release_as="0.1.0"', + tier: "step-run", + owner: "jobs.maintain step Pin only the first release to v0.1.0 run", + comparison: "contains", + }, + { + literal: "token: ${{ secrets.RELEASE_PLEASE_TOKEN }}", + tier: "structural", + owner: "jobs.maintain step Maintain release pull request with.token", + comparison: "equals ${{ secrets.RELEASE_PLEASE_TOKEN }}", + }, + { + literal: "release-as: ${{ steps.bootstrap-version.outputs.release_as }}", + tier: "structural", + owner: "jobs.maintain step Maintain release pull request with.release-as", + comparison: "equals ${{ steps.bootstrap-version.outputs.release_as }}", + }, + { + literal: "secrets.GITHUB_TOKEN", + tier: "structural", + owner: "jobs.maintain", + comparison: "absent from parsed job scalar values", + }, + { + literal: " needs:\n - resolve\n - package\n", + tier: "structural", + owner: "jobs.release.needs", + comparison: "deep equals [resolve, package]", + }, + { + literal: "group: release-publication-${{ needs.resolve.outputs.release_tag }}", + tier: "structural", + owner: "jobs.release.concurrency.group", + comparison: "equals release-publication-${{ needs.resolve.outputs.release_tag }}", + failureMessage: "release workflow publish mutation must serialize by resolved immutable tag", + }, + { + literal: + " permissions:\n actions: read\n contents: write\n id-token: write\n attestations: write\n issues: write\n pull-requests: write\n steps:\n", + tier: "structural", + owner: "jobs.release.permissions", + comparison: + "deep equals {actions: read, contents: write, id-token: write, attestations: write, issues: write, pull-requests: write}", + droppedAspect: + "Raw adjacency and ordering between the permissions block and steps is formatting, not parsed workflow behavior.", + }, +] as const satisfies readonly ReleaseWorkflowParityLedgerEntry[] + +type ActiveReleaseWorkflowParityEntry = Extract< + ReleaseWorkflowParityLedgerEntry, + { tier: ReleaseWorkflowParityTier } +> + +/** Fetch one object-shaped field from a parsed workflow record. */ +function releaseWorkflowRecordField( + record: Record | undefined, + field: string, +): Record | undefined { + const value = record?.[field] + return isRecord(value) ? value : undefined +} + +/** Fetch one scalar field from a named workflow step. */ +function releaseWorkflowStepField( + job: ReleaseWorkflowJob | undefined, + stepName: string, + field: string, +): unknown { + return releaseWorkflowStep(job, stepName)?.[field] +} + +/** Find every action step by its repository name, independent of the pinned ref. */ +function releaseWorkflowActionSteps( + job: ReleaseWorkflowJob | undefined, + actionName: string, +): ReleaseWorkflowStep[] { + return releaseWorkflowSteps(job).filter( + (step) => typeof step.uses === "string" && step.uses.split("@")[0] === actionName, + ) +} + +/** Find an action step by its repository name, independent of the pinned ref. */ +function releaseWorkflowActionStep( + job: ReleaseWorkflowJob | undefined, + actionName: string, +): ReleaseWorkflowStep | undefined { + return releaseWorkflowActionSteps(job, actionName)[0] +} + +/** Test every parsed scalar without depending on YAML formatting or comments. */ +function releaseWorkflowContainsScalar(value: unknown, literal: string): boolean { + if (typeof value === "string") return value.includes(literal) + if (Array.isArray(value)) { + return value.some((entry) => releaseWorkflowContainsScalar(entry, literal)) + } + if (isRecord(value)) { + return Object.values(value).some((entry) => releaseWorkflowContainsScalar(entry, literal)) + } + return false +} + +/** Map each tier-2 ledger owner to the job and step that own its run script. */ +const RELEASE_WORKFLOW_OWNED_STEPS: Record = + { + "jobs.resolve step Resolve unique candidate or immutable repair tag run": [ + "resolve", + "Resolve unique candidate or immutable repair tag", + ], + "jobs.maintain step Detect a merged release candidate stranded before its tag run": [ + "maintain", + "Detect a merged release candidate stranded before its tag", + ], + "jobs.package step Validate and prove release payload run": [ + "package", + "Validate and prove release payload", + ], + "jobs.package step Reject generated release-surface drift run": [ + "package", + "Reject generated release-surface drift", + ], + "jobs.release step Replay current publication admission before mutation run": [ + "release", + "Replay current publication admission before mutation", + ], + "jobs.release step Create or verify immutable tag run": [ + "release", + "Create or verify immutable tag", + ], + "jobs.release step Create missing GitHub Release and validate target run": [ + "release", + "Create missing GitHub Release and validate target", + ], + "jobs.release step Compare release assets before mutation run": [ + "release", + "Compare release assets before mutation", + ], + "jobs.release step Add or replace admitted release assets run": [ + "release", + "Add or replace admitted release assets", + ], + "jobs.compatibility step Prove packaged runtime custody on this target run": [ + "compatibility", + "Prove packaged runtime custody on this target", + ], + "jobs.package step Compare the rebuilt package with the platform-proven candidate run": [ + "package", + "Compare the rebuilt package with the platform-proven candidate", + ], + "jobs.release step Check for existing matching public attestation run": [ + "release", + "Check for existing matching public attestation", + ], + "jobs.maintain step Pin only the first release to v0.1.0 run": [ + "maintain", + "Pin only the first release to v0.1.0", + ], + } + +/** Resolve the parsed run script named by a tier-2 ledger owner. */ +function releaseWorkflowOwnedRun(workflow: unknown, owner: string): string { + const ownedStep = RELEASE_WORKFLOW_OWNED_STEPS[owner] + if (!ownedStep) { + throw new Error(`release workflow parity ledger has no step-run implementation for ${owner}`) + } + const run = releaseWorkflowStepField( + releaseWorkflowJob(workflow, ownedStep[0]), + ownedStep[1], + "run", + ) + return typeof run === "string" ? run : "" +} + +/** Match one tier-1 ledger entry against parsed workflow structure. */ +function releaseWorkflowStructuralEntryMatches( + workflow: unknown, + entry: ActiveReleaseWorkflowParityEntry, +): boolean { + const resolveJob = releaseWorkflowJob(workflow, "resolve") + const maintainJob = releaseWorkflowJob(workflow, "maintain") + const candidateJob = releaseWorkflowJob(workflow, "candidate") + const compatibilityJob = releaseWorkflowJob(workflow, "compatibility") + const packageJob = releaseWorkflowJob(workflow, "package") + const releaseJob = releaseWorkflowJob(workflow, "release") + const resolveStep = releaseWorkflowStep(resolveJob, "Resolve unique candidate or immutable repair tag") + const replayStep = releaseWorkflowStep( + releaseJob, + "Replay current publication admission before mutation", + ) + const packageUpload = releaseWorkflowActionStep(packageJob, "actions/upload-artifact") + const candidateUpload = releaseWorkflowActionStep(candidateJob, "actions/upload-artifact") + + switch (entry.owner) { + case "jobs.*.uses + jobs.*.steps[].uses": { + const references = releaseWorkflowActionReferences(workflow) + const externalReferences = references.filter( + (reference) => !releaseWorkflowLocalActionReference(reference), + ) + return ( + externalReferences.length > 0 && + externalReferences.every( + (reference) => + typeof reference === "string" && /^[^@\s]+@[a-f0-9]{40}$/.test(reference), + ) + ) + } + case "jobs.resolve step Resolve unique candidate or immutable repair tag env.EXPECTED_RELEASE_PLEASE_LOGIN": + return Object.hasOwn(releaseWorkflowRecordField(resolveStep, "env") ?? {}, "EXPECTED_RELEASE_PLEASE_LOGIN") + case "jobs.resolve step Resolve unique candidate or immutable repair tag env.PUSH_BEFORE_SHA": + return releaseWorkflowRecordField(resolveStep, "env")?.PUSH_BEFORE_SHA === "${{ github.event.before }}" + case "jobs.resolve step Resolve unique candidate or immutable repair tag env.PUSH_FORCED": + return releaseWorkflowRecordField(resolveStep, "env")?.PUSH_FORCED === "${{ github.event.forced }}" + case "jobs.resolve.outputs.merged_pr_base_sha": + case "jobs.resolve.outputs.reviewed_pr_head_sha": + case "jobs.resolve.outputs.trusted_base_sha": + return Object.hasOwn( + releaseWorkflowRecordField(resolveJob, "outputs") ?? {}, + entry.literal, + ) + case "jobs.maintain.steps[].name": + return releaseWorkflowSteps(maintainJob).some((step) => step.name === entry.literal) + case "jobs.compatibility.strategy.matrix.include[].runner": { + const strategy = releaseWorkflowRecordField(compatibilityJob, "strategy") + const matrix = releaseWorkflowRecordField(strategy, "matrix") + return ( + Array.isArray(matrix?.include) && + matrix.include.some((value) => isRecord(value) && value.runner === entry.literal) + ) + } + case "jobs.package step Validate and prove release payload env.SOURCE_COMMIT": + return Object.hasOwn( + releaseWorkflowRecordField( + releaseWorkflowStep(packageJob, "Validate and prove release payload"), + "env", + ) ?? {}, + "SOURCE_COMMIT", + ) + case "jobs.{candidate,compatibility,package,release} checkout steps with.ref": + return [candidateJob, compatibilityJob, packageJob, releaseJob].every((job) => { + const checkoutSteps = releaseWorkflowActionSteps(job, "actions/checkout") + return ( + checkoutSteps.length > 0 && + checkoutSteps.every( + (step) => + releaseWorkflowRecordField(step, "with")?.ref === + "${{ needs.resolve.outputs.candidate_sha }}", + ) + ) + }) + case "jobs.release step Replay current publication admission before mutation env.TRUSTED_BASE_SHA": + return releaseWorkflowRecordField(replayStep, "env")?.TRUSTED_BASE_SHA === "${{ needs.resolve.outputs.trusted_base_sha }}" + case "jobs.release step Replay current publication admission before mutation env.ADMITTED_MERGED_PR_BASE_SHA": + return releaseWorkflowRecordField(replayStep, "env")?.ADMITTED_MERGED_PR_BASE_SHA === "${{ needs.resolve.outputs.merged_pr_base_sha }}" + case "jobs.release step Replay current publication admission before mutation env.ADMITTED_REVIEWED_PR_HEAD_SHA": + return releaseWorkflowRecordField(replayStep, "env")?.ADMITTED_REVIEWED_PR_HEAD_SHA === "${{ needs.resolve.outputs.reviewed_pr_head_sha }}" + case "jobs.package upload-artifact step with.path": { + const path = releaseWorkflowRecordField(packageUpload, "with")?.path + return typeof path === "string" && path.includes(entry.literal) + } + case "on.workflow_dispatch.inputs.replace_mismatched_assets": { + const rootRecord = isRecord(workflow) ? workflow : undefined + const on = releaseWorkflowRecordField(rootRecord, "on") + const workflowDispatch = releaseWorkflowRecordField(on, "workflow_dispatch") + const inputs = releaseWorkflowRecordField(workflowDispatch, "inputs") + return Object.hasOwn(inputs ?? {}, "replace_mismatched_assets") + } + case "jobs.maintain.concurrency.group": + return releaseWorkflowRecordField(maintainJob, "concurrency")?.group === "release-maintenance" + case "jobs.release.concurrency.group": + return releaseWorkflowRecordField(releaseJob, "concurrency")?.group === "release-publication-${{ needs.resolve.outputs.release_tag }}" + case "jobs.package upload-artifact with.name and jobs.release download env.ARTIFACT_NAME": { + const producer = releaseWorkflowRecordField(packageUpload, "with")?.name + const consumer = releaseWorkflowRecordField( + releaseWorkflowStep(releaseJob, "Download proven release candidate"), + "env", + )?.ARTIFACT_NAME + return producer === entry.literal && consumer === producer + } + case "jobs.candidate upload-artifact with.name and jobs.{compatibility,package} download env.ARTIFACT_NAME": { + const producer = releaseWorkflowRecordField(candidateUpload, "with")?.name + const consumers = [ + releaseWorkflowStep(compatibilityJob, "Download the single packaged candidate"), + releaseWorkflowStep(packageJob, "Download the platform-proven release candidate"), + ].map((step) => releaseWorkflowRecordField(step, "env")?.ARTIFACT_NAME) + return producer === entry.literal && consumers.every((consumer) => consumer === producer) + } + case "jobs.{candidate,package} upload-artifact steps with.overwrite": + return [candidateUpload, packageUpload].every( + (step) => releaseWorkflowRecordField(step, "with")?.overwrite === true, + ) + case "jobs.release.environment": + return releaseJob?.environment === "release" + case "jobs.release step Add missing public release attestation uses": { + const uses = releaseWorkflowStep(releaseJob, "Add missing public release attestation")?.uses + return typeof uses === "string" && uses.split("@")[0] === entry.literal + } + case "jobs.release steps Check for existing matching public attestation and Add missing public release attestation if": + return [ + releaseWorkflowStep(releaseJob, "Check for existing matching public attestation"), + releaseWorkflowStep(releaseJob, "Add missing public release attestation"), + ].every((step) => typeof step?.if === "string" && step.if.includes(entry.literal)) + case "workflow.concurrency": + return isRecord(workflow) && !Object.hasOwn(workflow, "concurrency") + case "workflow (anywhere)": + return releaseWorkflowContainsScalar(workflow, entry.literal.replace(/^group: /, "")) + case "jobs.maintain": + return entry.literal === "secrets.GITHUB_TOKEN" + ? maintainJob !== undefined && !releaseWorkflowContainsScalar(maintainJob, entry.literal) + : maintainJob !== undefined + case "jobs.compatibility": + return compatibilityJob !== undefined + case "jobs.release": + return releaseJob !== undefined + case "jobs.converge": + return releaseWorkflowJob(workflow, "converge") !== undefined + case "jobs.maintain.concurrency.cancel-in-progress": + return releaseWorkflowRecordField(maintainJob, "concurrency")?.["cancel-in-progress"] === false + case "jobs.maintain checkout step with.persist-credentials": { + const maintainCheckoutSteps = releaseWorkflowActionSteps(maintainJob, "actions/checkout") + return ( + maintainCheckoutSteps.length > 0 && + maintainCheckoutSteps.every( + (step) => releaseWorkflowRecordField(step, "with")?.["persist-credentials"] === false, + ) + ) + } + case "jobs.maintain step Pin only the first release to v0.1.0 id": + return releaseWorkflowStep(maintainJob, "Pin only the first release to v0.1.0")?.id === "bootstrap-version" + case "jobs.maintain step Maintain release pull request with.token": + return releaseWorkflowRecordField( + releaseWorkflowStep(maintainJob, "Maintain release pull request"), + "with", + )?.token === "${{ secrets.RELEASE_PLEASE_TOKEN }}" + case "jobs.maintain step Maintain release pull request with.release-as": + return releaseWorkflowRecordField( + releaseWorkflowStep(maintainJob, "Maintain release pull request"), + "with", + )?.["release-as"] === "${{ steps.bootstrap-version.outputs.release_as }}" + case "jobs.release.needs": + return ( + Array.isArray(releaseJob?.needs) && + releaseJob.needs.length === 2 && + releaseJob.needs[0] === "resolve" && + releaseJob.needs[1] === "package" + ) + case "jobs.release.permissions": { + const permissions = releaseWorkflowRecordField(releaseJob, "permissions") + const expected = { + actions: "read", + contents: "write", + "id-token": "write", + attestations: "write", + issues: "write", + "pull-requests": "write", + } + return ( + permissions !== undefined && + Object.keys(permissions).length === Object.keys(expected).length && + Object.entries(expected).every(([key, value]) => permissions[key] === value) + ) + } + default: + throw new Error( + `release workflow parity ledger has no structural implementation for ${entry.owner}`, + ) + } +} + +/** Preserve the validator's established diagnostics for ledger assertion failures. */ +function releaseWorkflowParityError(entry: ActiveReleaseWorkflowParityEntry): string { + if (entry.failureMessage !== undefined) return entry.failureMessage + if (entry.owner === "jobs.*.uses + jobs.*.steps[].uses") { + return "release workflow actions must be pinned to full commit SHAs" + } + if (entry.literal === "parent_count" || entry.literal === "mergeMode") { + return `release workflow retains unsupported merge-shape metadata: ${entry.literal}` + } + if (entry.literal === "github.run_attempt") { + return "release workflow artifact identity must survive rerun-failed-jobs attempts" + } + if (entry.owner === "workflow.concurrency") { + return "release workflow must serialize only mutation jobs, not discard distinct pending runs" + } + if ( + entry.literal === "group: release-maintenance" || + entry.literal === "group: release-publication-${{ needs.resolve.outputs.release_tag }}" + ) { + return `release workflow is missing ${entry.literal}` + } + if (entry.owner === "jobs.maintain" || entry.owner === "jobs.compatibility") { + if (entry.literal === "secrets.GITHUB_TOKEN") { + return "release workflow maintenance job must not fall back to GITHUB_TOKEN" + } + return "release workflow is missing the maintain or compatibility job boundary" + } + if (entry.owner === "jobs.release") return "release workflow is missing the release job boundary" + if (entry.owner === "jobs.converge") return "release workflow is missing the converge job boundary" + if ( + [ + "cancel-in-progress: false", + "persist-credentials: false", + "id: bootstrap-version", + "jq 'length' .github/.release-please-manifest.json", + 'release_as="0.1.0"', + "token: ${{ secrets.RELEASE_PLEASE_TOKEN }}", + "release-as: ${{ steps.bootstrap-version.outputs.release_as }}", + ].includes(entry.literal) + ) { + return `release workflow maintenance job is missing ${entry.literal}` + } + if (entry.owner === "jobs.release.needs") { + return "release workflow publish job must depend on package" + } + if (entry.owner === "jobs.release.permissions") { + return "release workflow publish job permissions must match the protected release contract" + } + return `release workflow is missing ${entry.literal}` +} + +/** Implement every non-drop parity-ledger entry at its recorded assertion tier. */ +export function validateReleaseWorkflowParity(workflowSource: string, workflow: unknown): void { + const jobNames = Object.keys(releaseWorkflowJobs(workflow)) + const maintainPosition = jobNames.indexOf("maintain") + const compatibilityPosition = jobNames.indexOf("compatibility") + if ( + releaseWorkflowJob(workflow, "maintain") === undefined || + releaseWorkflowJob(workflow, "compatibility") === undefined || + compatibilityPosition <= maintainPosition + ) { + throw new Error("release workflow is missing the maintain or compatibility job boundary") + } + const releasePosition = jobNames.indexOf("release") + const convergePosition = jobNames.indexOf("converge") + if (releaseWorkflowJob(workflow, "release") === undefined) { + throw new Error("release workflow is missing the release job boundary") + } + if (releaseWorkflowJob(workflow, "converge") === undefined) { + throw new Error("release workflow is missing the converge job boundary") + } + if (convergePosition <= releasePosition) { + throw new Error("release workflow converge job must follow the release job") + } + + for (const entry of RELEASE_WORKFLOW_PARITY_LEDGER) { + if ("dropReason" in entry) continue + let matches: boolean + switch (entry.tier) { + case "structural": + matches = releaseWorkflowStructuralEntryMatches(workflow, entry) + break + case "step-run": + matches = releaseWorkflowOwnedRun(workflow, entry.owner).includes(entry.literal) + break + case "raw-residual": + matches = !workflowSource.includes(entry.literal) + break + } + if (!matches) throw new Error(releaseWorkflowParityError(entry)) + } +}