fix(evidence): make branch order and check identity representable - #41
Merged
Merged
Conversation
Bundle schema 1.0 could not answer the question CM-2.A2 exists to ask. Canonicalization sorts commits by authored_at, which destroys branch order, and parents were never collected -- so "which commits landed after this approval" was uncomputable. Author dates also survive rebase and cherry-pick, so the timestamp fallback read a rebased branch as freshly approved. derived.last_production_commit_at compounded it: the name claimed a scoping that never happened (it took production_paths and ignored them) and returned max(authored_at) over every commit, so a docs-only follow-up flipped a good approval to stale. Separately, checks tied on (name, source) and carried no id. Re-running a workflow produced two entries that sorted by whatever the API returned that minute, so identical evidence could hash differently between runs -- the same silent shape as the three Stage 0 bugs. The tiebreak is now the item's own canonical form, applied to every array, so ordering depends on contents alone. Schema 1.0 -> 1.1. ENGINE_VERSION 0.0.1 -> 0.1.0.
Tickmark — evidence collectedControl evaluation is not wired up yet. This is the evidence the engine sees.
Notices
Provenance
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Bundle schema 1.0 → 1.1,
ENGINE_VERSION0.0.1 → 0.1.0.commits[].sequence— position GitHub returned the commit in, so branch order survives theauthored_atsort applied during canonicalization.commits[].parents— deliberately unsorted; parent order is semantic in git. Lets branch order be verified rather than believed.checks[].id— a re-run repeats the name for the same SHA; without an id there is no "current result for this check".derived.last_production_commit_atremoved. Replaced bycommit_shas_in_order,commit_order_verified,head_commit_sha,last_commit_at(committer date),last_authored_at.checks.Why this approach
The obvious alternative was to leave the bundle alone and build CM-2.A2 on
max(authored_at). That ships a control I already know misreads any rebased branch, and the corpus is not yet frozen, so this is the cheapest this change will ever be.Three things were wrong at once:
Branch order was unrecoverable. Sorting commits by
authored_atis correct for hashing and destructive for reasoning.sequencecarries the order;parentsmakes it checkable. When the chain does not walk — force-push, absent parents, unusual merge shape —commit_order_verifiedis false and a dependent check reports INDETERMINATE rather than guessing (AGENTS.md rule 5).last_production_commit_atlied three ways. It tookproduction_pathsand never applied it, so the name claimed a scoping that did not happen and a docs-only follow-up commit flipped a good approval to stale. And author date is preserved by rebase, amend, and cherry-pick, so a rebased branch read as freshly approved. Names now say what the values are.checkshad no unique tiebreak. Python's sort is stable, so ties fall back to input order, and input order is the GitHub API's — which promises nothing. Two check runs from a re-run tie on(name, source). Identical evidence could hash differently between runs, with nothing raised. Same shape as all three Stage 0 bugs. Fixed by appending the canonical form as final tiebreak to every array, so the failure mode closes rather than the instance._max_stampalso filters to the canonical timestamp shape. Canonicalization passes an unparseable timestamp through verbatim by design, and a raw stringmax()would return it — letters sort above digits. A mangled input now yieldsNone, so a check sees unknown and reports unknown.Migration note
No migration path. Nothing consumed 1.0 outside this repository and the eval corpus is not frozen.
ENGINE_VERSIONbumped so any cached verdict computed under the old derivation is correctly invalidated. Schema history recorded indocs/technical-design.md§1.How it was verified
TestCommitOrder,TestCommitTimestamps(test_derive),TestArrayOrdering(test_canonical),TestCommitShaping(test_collector).make testandmake linton 3.10 and 3.12 via CI.