fix(context+checkpoints): the deterministic-derivation identity enters the resume keys — source_sha256 stamped by all three context arms, folded into the gated phases' fingerprints (#546) - #572
Conversation
…s the resume keys — source_sha256 stamped by all three context arms, folded into the gated phases' fingerprints (#546) The checkpoint family's resume keys excluded the app-context payload — correctly for the NARRATED text (the LLM non-determinism trap), but the context is derived from deterministic repo sources: a repo edit changing the derivation while leaving a unit's code identical replayed stale signals under the changed analysis context (they gate the re-filter — the FN-risk class the two-seat gate on PR #543 named). The fix (the need-check's stamp-the-artifact architecture — no consumer re-derivation, no repo_path plumbing, no production/consumption race): - context_sources_digest: the canonical sha over the gathered CONTEXT_FILES entries + the two degradation markers — deliberately EXCLUDING [directory_structure] (it lists the in-repo output dir; hashing it self-invalidates on every resume that writes an artifact) and [detected_patterns] (rglob-order, cap-windowed — not canonical); those stay a named residual; - all THREE arms stamp source_sha256 on the artifact: the threat-model arm (the file's bytes — pre-existing), the manual-override arm (the override content), the LLM arm (the sources digest) — a DIFFERENT narration over the same sources keeps the stamp; - the gated phases fold it into their identity keys via extra_key (ctx_sources_sha256): analyze (the load-before-fingerprint reorder — the prior order keyed on None forever), verify (the standalone --app-context case), llm-reach (the dir-level fingerprint; the per-prompt narration stays excluded). A pre-fix artifact yields None (one-time archive). - FINGERPRINT_SCHEME_VERSION 2 + the exclusion clause rewritten: the NARRATION never enters; the DERIVATION identity does.
…h producer arms overwrite a supplied identity (#546 review round) The review round's request-changes: the LLM arm and the override arm honored a SUPPLIED source_sha256 (if not data.get(...)) — a scanned repo could steer the model's JSON (the sources text is attacker-influenceable) or author an OPENANT.json that pins a constant sha, freezing the checkpoint identity against future source edits — re-opening exactly the stale-replay #546 exists to kill. Both arms now compute UNCONDITIONALLY (the LLM arm: context_sources_digest(sources); the override arm: sha over the serialized override data). Negative tests: two different supplied constants + the same sources yield the same DERIVED digest (not either constant); a override-supplied sha is overwritten. Evidence: pytest tests/test_issue546_ctx_fingerprint.py tests/test_issue293_three_state_counters.py — 24 passed; full suite — 4030 passed, 33 skipped, 0 FAIL.
gadievron
left a comment
There was a problem hiding this comment.
Verification review — recommend-merge. (all evidence pinned to head 3f6813b; re-verify on any new head)
The fix: context_sources_digest — the canonical sha over the gathered CONTEXT sources (excluding [directory_structure], which self-invalidates, and [detected_patterns], which is rglob-order-dependent — a named residual); all three context arms stamp source_sha256; the three identity-gate sites (analyze, verify, LLR) fold ctx_sources_sha256 into their fingerprints; FINGERPRINT_SCHEME_VERSION 1→2; the app context now loads BEFORE the analyze fingerprint. The narration stays excluded (the verified ~17k-token re-pay clause is amended, not violated) — only the deterministic derivation invalidates.
The review round, fixed on this branch (3f6813b): the LLM arm and the override arm HONORED a supplied source_sha256 (if not data.get(...)) — a scanned repo could steer the model's JSON (the sources text is attacker-influenceable) or author an OPENANT.json that pins a constant sha, freezing the checkpoint identity against future source edits — re-opening exactly the stale replay #546 exists to kill. Both arms now compute UNCONDITIONALLY (LLM: the sources digest; override: sha over the serialized override data). Negative tests pin it: two different supplied constants + the same sources yield the same DERIVED digest (neither constant adopted).
Verification (run at the pinned head): targeted slices (issue546 + issue293) — 24 passed; full suite — 4030 passed, 33 skipped, 0 FAIL; CI at 3f6813b: tests + gitleaks success. Cross-PR: the combined tree of all six open PRs passed the full suite; the #572×#575 same-file (llm_reachability.py, disjoint regions) merge is clean; this is the only PR bumping the scheme version.
Known cost (for the approver): the scheme bump 1→2 invalidates every pre-existing analyze/verify/LLR sidecar — a one-time re-pay for all users, including no-context users whose old checkpoints were valid (policy-conformant per backend_identity's own bump rule; not strictly correctness-required, since a post-fix scan's key already differs).
Follow-ups (named, none blocking): the verifier/LLR fold sites have no dedicated wiring tests (only _analyze_fingerprint is pinned); the LLR site reads the artifact dict while analyze/verify read the dataclass attribute (type asymmetry — safe today since the scanner passes the raw dict); a YAML override with a non-JSON-serializable scalar silently degrades source_sha256 to None and skips the fold for that run (json.dumps(..., default=str) would keep it derived); unhashed renames that flip detected_patterns classification remain the named residual.
Approve + merge are code-owner actions; never on red CI.
Summary
The checkpoint family's resume keys excluded the app-context payload — correctly for the narrated text (the LLM non-determinism trap), but the context is derived from deterministic repo sources: a repo edit changing the derivation while leaving a unit's code identical replayed stale signals under the changed analysis context (they gate the re-filter — the FN-risk class the two-seat gate on PR #543 named).
The fix (the stamp-the-artifact architecture — no consumer re-derivation, no
repo_pathplumbing, no production/consumption race):context_sources_digest: the canonical sha over the gathered CONTEXT_FILES entries + the two degradation markers — deliberately excluding[directory_structure](it lists the in-repo output dir; hashing it would self-invalidate on every resume that writes an artifact) and[detected_patterns](rglob-order, cap-windowed — not canonical); those stay a named residual;source_sha256on the artifact: the threat-model arm (the file's bytes — pre-existing), the manual-override arm (the override content), the LLM arm (the sources digest) — a different narration over the same sources keeps the stamp (no spurious re-pay);extra_key(ctx_sources_sha256): analyze (the load-before-fingerprint reorder — the prior order computed the fingerprint before the context loaded, keying onNoneforever), verify (the standalone--app-contextcase), llm-reach (the dir-level fingerprint; the per-prompt narration stays excluded). A pre-fix artifact yieldsNone→ a one-time archive on first contact;FINGERPRINT_SCHEME_VERSION→ 2 + the exclusion clause rewritten: the narration never enters; the derivation identity does.Closes #546.
Test plan
9 tests: the digest (same-files/same, content-change differs, new-context-file differs, output-dir artifacts do not invalidate — the self-invalidation hazard, untracked-py-under-results unchanged); the producer stamps (a source landing between calls differs; a different narration over the same sources keeps the stamp); the analyze fingerprint folds the sha (None vs sha differ; the pre-fix artifact loads yielding None).
Verification evidence
pytest tests/test_issue546_ctx_fingerprint.py -qpytest tests/test_issue293_three_state_counters.py tests/test_issue532_llr_resume.py tests/test_llm_reachability.py tests/test_backend_identity.py -qpytest tests/ -qgather_context_sourceshash rejected (the[directory_structure]self-invalidation + the[detected_patterns]order hazard); the consumer-side re-derivation rejected (therepo_pathcrux); the three-arm coverage gap (threat-model + manual-override arms never callgather_context_sources) closed by the stamp-the-artifact design