feat(roms): datfile-as-dependency pin + fetch + SHA-256 verify (B-0083.1)

docs/BACKLOG.md

@@ -54,6 +54,7 @@ are closed (status: closed in frontmatter)._
 - [x] **[B-0066](backlog/P1/B-0066-memory-md-marker-vs-index-harness-verify-q1-automemory-aaron-2026-04-28.md)** MEMORY.md marker-vs-index — verify harness contract + Q1 AutoDream/AutoMemory compatibility, then migrate (Aaron 2026-04-28)
 - [x] **[B-0067](backlog/P1/B-0067-cadenced-git-hotspot-detection-aaron-2026-04-28.md)** Cadenced git-hotspot detection — all 3 phases complete
 - [ ] **[B-0083](backlog/P1/B-0083-atari-2600-rom-canonical-naming-tosec-goodtools-tooling-aaron-2026-04-28.md)** Atari 2600 ROM canonical-naming + safe-vs-unsafe folder split + TOSEC/Good-Tools-style hash-lookup tooling
+- [ ] **[B-0083.1](backlog/P1/B-0083.1-atari-rom-datfile-as-dependency-pin-fetch-verify-2026-05-29.md)** ROM datfile-as-dependency — pin + fetch + SHA-256 verify + refresh
 - [ ] **[B-0087](backlog/P1/B-0087-github-settings-drift-workflow-broken-invalid-permission-administration-otto-2026-04-28.md)** github-settings-drift.yml broken since PR #45 — option A landed (invalid permission removed); options B/C remain maintainer-gated
 - [x] **[B-0110](backlog/P1/B-0110-acehack-mirror-protocol-drift-2026-04-30.md)** AceHack mirror-refresh protocol drift — Path 2 chosen, doctrine update landing in same PR (2026-04-30)
 - [x] **[B-0125](backlog/P1/B-0125-skip-fsharp-analyze-on-docs-only-prs-2026-05-01.md)** Skip F#/Analyze (csharp) on docs-only PRs without tripping `code_quality severity:all`

docs/backlog/P1/B-0083-atari-2600-rom-canonical-naming-tosec-goodtools-tooling-aaron-2026-04-28.md

@@ -7,16 +7,30 @@ tier: factory-tooling
 effort: M
 ask: maintainer Aaron 2026-04-28 (autonomous-loop ROM-drop + canonical-naming request)
 created: 2026-04-28
-last_updated: 2026-05-02
+last_updated: 2026-05-29
 decomposition: decomposed
-children: [B-0272, B-0273]
+children: [B-0272, B-0273, B-0083.1]
 depends_on: []
 tags: [aaron-2026-04-28, roms, atari-2600, tosec, good-tools, canonical-naming, datfile, license-safety, gitignore-already-protects, high-priority-after-0-0-0, scheduled-after-0-0-0]
 type: friction-reducer
 ---
 
 # B-0083 — Atari 2600 ROM canonical-naming + tooling
 
+## Decomposition status (2026-05-29)
+
+- **B-0272** (canonical naming via TOSEC/No-Intro hash lookup) — **closed**
+  2026-05-16. `tools/roms/canonicalize.ts` (+ tests).
+- **B-0273** (safe/unsafe folder split) — **closed** 2026-05-29.
+  `tools/roms/split-by-license.ts` + `roms-safe/atari/2600/` + allowlist +
+  README cross-refs.
+- **B-0083.1** (datfile-as-dependency: pin + fetch + SHA-256 verify + refresh)
+  — **open**. The parent acceptance criterion #6 ("Tooling refreshes on TOSEC
+  datfile updates") + the "Datfile-as-dependency" design section were not
+  covered by either earlier child; B-0083.1 builds the pin manifest +
+  fetch-and-verify tool. Parent stays open until B-0083.1's operator-fill +
+  optional-refresh-cadence steps resolve.
+
 ## Source
 
 Aaron 2026-04-28T18:55Z verbatim:

docs/backlog/P1/B-0083.1-atari-rom-datfile-as-dependency-pin-fetch-verify-2026-05-29.md

@@ -0,0 +1,83 @@
+---
+id: B-0083.1
+priority: P1
+status: open
+title: "ROM datfile-as-dependency — pin + fetch + SHA-256 verify + refresh"
+created: 2026-05-29
+last_updated: 2026-05-29
+parent: B-0083
+depends_on: [B-0272]
+classification: buildable-now
+decomposition: atomic
+type: friction-reducer
+tags: [roms, atari-2600, tosec, datfile, dep-pin, fetch-verify, otto-247]
+---
+
+# B-0083.1 — ROM datfile-as-dependency
+
+The third slice the parent B-0083 acceptance list named but neither closed
+sibling covered. B-0272 built `canonicalize.ts` (which *consumes*
+`--datfile <path>`) and B-0273 built `split-by-license.ts` + `roms-safe/`.
+Neither built the mechanism that *produces* the pinned, verified datfile.
+
+This row operationalizes the parent's **"Datfile-as-dependency"** design
+section + the **"Tooling refreshes on TOSEC datfile updates"** acceptance
+criterion: pin a datfile version, download it, verify its SHA-256, refresh
+on update.
+
+## Pre-start checklist
+
+- [x] Prior-art search: read parent B-0083 (algorithm + "Datfile-as-dependency"
+  + "Recommended approach"); both children B-0272 (closed) + B-0273 (closed);
+  `tools/roms/canonicalize.ts` (`--datfile` consumer) + `split-by-license.ts`;
+  grepped `tools/` + `.github/workflows/` for any datfile fetch/pin/refresh
+  tooling — none existed.
+- [x] Dependency walk: depends on B-0272 (the consumer of the fetched datfile,
+  already closed). Sibling B-0273 closed. No other deps.
+- [x] Otto-247 / dep-pin: WebSearch'd "TOSEC Atari 2600 datfile latest" —
+  latest TOSEC release is **2025-03-13** (no 2026 release as of 2026-05).
+  Sources: <https://www.tosecdev.org/news/releases/177-tosec-release-2025-03-13>,
+  <https://mail.tosecdev.org/downloads/category/59-2025-03-13>.
+
+## Acceptance criteria
+
+- [x] Structured pin manifest at `tools/roms/manifests/datfiles.json`
+  (platform → source/release/datfileName/sourceUrl/downloadUrl/sha256).
+- [x] Fetch-and-verify tool at `tools/roms/fetch-datfile.ts`: download the
+  pinned datfile, SHA-256-verify against the pin, write to a gitignored
+  cache (`roms/.datfiles/`), emit the `canonicalize.ts --datfile` follow-up.
+- [x] **Fail-closed** on unverified pins: any `<...>` placeholder
+  (`downloadUrl`/`sha256`) is refused (exit 2), never written — per
+  `.claude/rules/dep-pin-search-first-authority.md`.
+- [x] `--list` surfaces pinned platforms + verification status.
+- [x] Tests (`fetch-datfile.test.ts`) for manifest parse, placeholder gate,
+  SHA-256 verify, and CLI error paths — no network in tests.
+- [x] `atari-2600` pinned to the WebSearch-verified TOSEC 2025-03-13 release,
+  with `downloadUrl` + `sha256` as explicit fail-closed placeholders.
+- [ ] **Operator action (not code)**: on first network-enabled run, fill the
+  verified `downloadUrl` + `sha256` for the atari-2600 datfile in the
+  manifest (the tool's refusal message names the exact steps). This is the
+  one value that requires a real download to verify and so could not be
+  pinned at authoring time.
+- [ ] **Optional / deferred**: a scheduled GHA refresh cadence (manual-trigger
+  workflow). Out of scope for this slice; the parent marks it "scheduled cron
+  optional". File a follow-up if/when a refresh cadence is wanted.
+
+## Why this is the smallest safe slice of B-0083
+
+Both decomposed children (B-0272, B-0273) are closed. The parent's only
+genuinely-unbuilt, repo-shippable acceptance criterion was the
+datfile-as-dependency mechanism (the "rename all 3461 ROMs" criterion is a
+local-data op on gitignored files, and the README cross-ref already landed in
+B-0273). This slice closes that gap with a bounded, tested tool that composes
+with the existing `canonicalize.ts --datfile` consumer.
+
+## Composes with
+
+- `tools/roms/canonicalize.ts` (B-0272) — consumes the fetched + verified datfile.
+- `tools/roms/split-by-license.ts` (B-0273) — downstream of canonicalization.
+- `.claude/rules/dep-pin-search-first-authority.md` — the fail-closed placeholder
+  pattern + WebSearch-verified pin discipline.
+- `roms/.gitignore` — `roms/.datfiles/` cache is gitignored by the existing
+  depth-limited rule (only READMEs tracked).
+- Parent B-0083 acceptance criterion #6 + "Datfile-as-dependency" design section.

tools/roms/fetch-datfile.test.ts

@@ -0,0 +1,190 @@
+import { describe, expect, test } from "bun:test";
+import { mkdtempSync, writeFileSync } from "node:fs";
+import { join } from "node:path";
+import { tmpdir } from "node:os";
+import {
+  parseManifest,
+  isPlaceholder,
+  sha256Hex,
+  verifyChecksum,
+  fetchBlockReason,
+  loadManifest,
+  main,
+  DEFAULT_MANIFEST,
+  type DatfilePin,
+} from "./fetch-datfile.ts";
+
+const VERIFIED_PIN: DatfilePin = {
+  platform: "test-2600",
+  source: "TOSEC",
+  release: "2025-03-13",
+  datfileName: "Test (TOSEC).dat",
+  sourceUrl: "https://example.test/release",
+  // sha256Hex of UTF-8 "abc"
+  downloadUrl: "https://example.test/test.dat",
+  sha256: "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad",
+  romsDir: "test/2600",
+};
+
+function manifestJson(pins: readonly Partial<DatfilePin>[]): string {
+  return JSON.stringify({ datfiles: pins });
+}
+
+describe("parseManifest", () => {
+  test("parses a well-formed entry into a platform-keyed map", () => {
+    const pins = parseManifest(manifestJson([VERIFIED_PIN]));
+    expect(pins.size).toBe(1);
+    expect(pins.get("test-2600")?.source).toBe("TOSEC");
+  });
+
+  test("throws on invalid JSON", () => {
+    expect(() => parseManifest("{not json")).toThrow(/not valid JSON/);
+  });
+
+  test("throws when datfiles array is missing", () => {
+    expect(() => parseManifest("{}")).toThrow(/missing "datfiles"/);
+  });
+
+  test("throws on an entry missing a required field", () => {
+    const { sha256, ...partial } = VERIFIED_PIN;
+    expect(() => parseManifest(manifestJson([partial]))).toThrow(/sha256/);
+  });
+
+  test("throws on an empty-string required field", () => {
+    expect(() =>
+      parseManifest(manifestJson([{ ...VERIFIED_PIN, platform: "" }])),
+    ).toThrow(/platform/);
+  });
+
+  test("throws on a missing romsDir (explicit-path field is required)", () => {
+    const { romsDir, ...partial } = VERIFIED_PIN;
+    expect(() => parseManifest(manifestJson([partial]))).toThrow(/romsDir/);
+  });
+
+  test("throws on a duplicate platform", () => {
+    expect(() =>
+      parseManifest(manifestJson([VERIFIED_PIN, VERIFIED_PIN])),
+    ).toThrow(/duplicate platform/);
+  });
+
+  test("ignores the _comment field", () => {
+    const json = JSON.stringify({
+      _comment: ["doc string"],
+      datfiles: [VERIFIED_PIN],
+    });
+    expect(parseManifest(json).size).toBe(1);
+  });
+});
+
+describe("isPlaceholder", () => {
+  test("flags angle-bracket-wrapped values", () => {
+    expect(isPlaceholder("<SHA256-VERIFY-ON-FETCH>")).toBe(true);
+    expect(isPlaceholder("<DIRECT-DAT-URL-VERIFY-ON-FETCH>")).toBe(true);
+  });
+
+  test("passes concrete values", () => {
+    expect(isPlaceholder("https://example.test/x.dat")).toBe(false);
+    expect(isPlaceholder("ba7816bf")).toBe(false);
+    expect(isPlaceholder("")).toBe(false);
+  });
+});
+
+describe("sha256Hex / verifyChecksum", () => {
+  const abc = new TextEncoder().encode("abc");
+  const knownAbc =
+    "ba7816bf8f01cfea414140de5dae2223b00361a396177a9cb410ff61f20015ad";
+
+  test("computes a known SHA-256", () => {
+    expect(sha256Hex(abc)).toBe(knownAbc);
+  });
+
+  test("verifyChecksum accepts a match (case-insensitive expected)", () => {
+    expect(verifyChecksum(abc, knownAbc)).toBe(true);
+    expect(verifyChecksum(abc, knownAbc.toUpperCase())).toBe(true);
+  });
+
+  test("verifyChecksum rejects a mismatch", () => {
+    expect(verifyChecksum(abc, "deadbeef")).toBe(false);
+    expect(verifyChecksum(new TextEncoder().encode("abd"), knownAbc)).toBe(
+      false,
+    );
+  });
+});
+
+describe("fetchBlockReason (fail-closed gate)", () => {
+  test("blocks a placeholder downloadUrl", () => {
+    const pin = { ...VERIFIED_PIN, downloadUrl: "<DIRECT-DAT-URL-VERIFY-ON-FETCH>" };
+    expect(fetchBlockReason(pin)).toMatch(/downloadUrl.*placeholder/);
+  });
+
+  test("blocks a placeholder sha256", () => {
+    const pin = { ...VERIFIED_PIN, sha256: "<SHA256-VERIFY-ON-FETCH>" };
+    expect(fetchBlockReason(pin)).toMatch(/sha256.*placeholder/);
+  });
+
+  test("allows a fully verified pin", () => {
+    expect(fetchBlockReason(VERIFIED_PIN)).toBeNull();
+  });
+});
+
+describe("real pinned manifest", () => {
+  test("tools/roms/manifests/datfiles.json parses", () => {
+    const pins = loadManifest(DEFAULT_MANIFEST);
+    expect(pins.has("atari-2600")).toBe(true);
+  });
+
+  test("atari-2600 pins the WebSearch-verified TOSEC release", () => {
+    const pin = loadManifest(DEFAULT_MANIFEST).get("atari-2600")!;
+    expect(pin.source).toBe("TOSEC");
+    expect(pin.release).toBe("2025-03-13");
+  });
+
+  test("atari-2600 is fail-closed until the operator verifies the pin", () => {
+    const pin = loadManifest(DEFAULT_MANIFEST).get("atari-2600")!;
+    expect(fetchBlockReason(pin)).not.toBeNull();
+  });
+});
+
+describe("main (CLI, no network)", () => {
+  test("no args => usage error exit 64", async () => {
+    expect(await main([])).toBe(64);
+  });
+
+  test("unknown arg => exit 64", async () => {
+    expect(await main(["--bogus"])).toBe(64);
+  });
+
+  test("--help => exit 0", async () => {
+    expect(await main(["--help"])).toBe(0);
+  });
+
+  test("missing manifest => exit 1", async () => {
+    expect(
+      await main(["--platform", "atari-2600", "--manifest", "/no/such/file"]),
+    ).toBe(1);
+  });
+
+  test("--list against real manifest => exit 0", async () => {
+    expect(await main(["--list"])).toBe(0);
+  });
+
+  test("unknown platform => exit 1", async () => {
+    expect(await main(["--platform", "nonesuch-9999"])).toBe(1);
+  });
+
+  test("placeholder pin fails closed => exit 2 (no network)", async () => {
+    expect(await main(["--platform", "atari-2600"])).toBe(2);
+  });
+
+  test("verified pin with missing platform in a temp manifest still exit 2 when placeholder", async () => {
+    const dir = mkdtempSync(join(tmpdir(), "datfile-test-"));
+    const path = join(dir, "m.json");
+    writeFileSync(
+      path,
+      manifestJson([{ ...VERIFIED_PIN, downloadUrl: "<placeholder>" }]),
+    );
+    expect(
+      await main(["--platform", "test-2600", "--manifest", path]),
+    ).toBe(2);
+  });
+});