feat(recall): entire recall — checkpoint history as trust-ranked associative memory

[voxmastery]

Summary

Adds an experimental entire recall command: checkpoint history as trust-ranked associative memory. A developer, or a fresh agent session whose context window has closed, asks "what did we decide and why" and gets ranked hits, each carrying a tier (LEDGER commit record / INTENT user prompt / chat assistant claim), a verdict of the claim against the commit it sits on, and the backing commit — instead of a transcript dump.

Git records what changed; checkpoints preserve why. But preserved context is an archive, not a memory: search returns transcripts, and a statement like "this change is isolated to one package" sits there looking as authoritative as everything else even when the diff reaches three others. recall ranks checkpoint context by how much it can be trusted, not by keyword match.

Built on Track 1 of the BTW Buildathon 2026 (Bengaluru) with Entire's own checkpoints, trailers and Graph as the input corpus. This PR is the feature only; the event write-up, benchmark provenance and demo live on the fork's main.

What is in the PR

Go shim — cmd/entire/cli/recall_cmd.go, recall_collect.go (+ tests). entire recall ingest walks trailered commits on the current branch, reads each checkpoint's transcript and diff through the persistent store, and pipes JSON to the recall binary. entire recall <question> renders the ranked hits (--json for machines). No ranking logic lives in Go.

Rust crate — recall/ on FluctlightDB (embedded, local, MIT/Apache-2.0). ingest maps prompts (UserExplicit), assistant turns (ChatAssertion), commit records (LedgerVerified) and touched files to engrams. activate never sorts on raw activation: provenance → SourceKind; agreement() runs six ordered checks of a claim against its commit — graph-reach scope, named file, action polarity, identifier anchor, lexical, 4-gram cosine — and the result adds or removes evidence before a noisy-OR confidence sets the score.

Entire Graph supplies the reach the scope check uses: one entire graph symbols stream, then impact on the first two code symbols each touched file defines, capped at 40 queries per ingest. No graph → no edges; nothing is fabricated.

Wiring follows the existing conventions: experimental.Register (hidden in stable builds, shown in developer/nightly), an agentHelpClassification entry (recall and recall ingest are task-driven, unlisted — ingest rebuilds derived state under .entire/recall, so the group cannot claim read-only), a row in entire labs, and a CLAUDE.md paragraph. The binary is located via $ENTIRE_RECALL_BIN, then recall on $PATH, then recall/target/{release,debug}/recall; the Go side builds and tests without Rust installed.

Privacy boundary: sensitive repositories

• Nothing leaves the machine. The brain is an embedded database under .entire/recall/ (gitignored). The only network on the ingest path is git fetching the repo's own checkpoint refs. recall/scripts/verify-offline.sh runs ingest and activate under strace -f -e trace=network and fails on a single socket call — measured, not asserted, because FluctlightDB's dependency tree does include hyper/tokio (behind an unused feature and an uncalled server module).
• A checkpoint whose transcript cannot be read is never dropped. Its commit record is ingested with the missing fields declared in unavailable. On this repository that is the difference between 3 and 125 checkpoints, because most checkpoint refs live on an upstream remote a fork cannot read.
• Fourth verdict unverifiable, distinct from neutral. Neutral means "checked, found nothing". Unverifiable means "could not check: the field this check needed was redacted or unavailable". Unavailability is declared by the shim, never inferred from an empty field, so complete checkpoints take a byte-identical path.
• The interface never presents partial context as complete. Every answer opens with a coverage line; --json returns {"coverage": ..., "hits": [...]}. complete is printed only when every checkpoint was indexed whole, the commit walk was not cut short, and the graph was consulted. Hits from partial context carry partial: [...] and their confidence is capped at 0.50, below any corroborated hit.
• entire recall ingest --no-transcripts never opens a transcript.

Evaluation

cargo run --release --example bench runs agreement() over 840 labelled claim/commit pairs generated from this repository's own public history under four of the CodeFuse-CommitEval mutation rules (operation type, file path, function name, component):

metric	value
Precision	0.869
Recall	0.270
Specificity	0.927
Latency	~145 µs per verdict

Precision-biased on purpose: when it flags a claim as contradicted it is right 87% of the time, and it stays silent otherwise. Recall is the documented limitation. The thresholded checks are calibrated to this repo's commit style; the structural identifier-anchor check is the part that transfers.

Note on recall/bench/bench_840.json (3.7 MB): it contains real diffs from this repository's history, including the AWS Access Key ID fixture your secret-redaction tests use. GitHub secret scanning flags it on the fork; it is your own test fixture, already present across many upstream commits, not a live credential. Happy to move the file to Git LFS or an external download if you would rather not carry it in-tree.

Test plan

• go test ./cmd/entire/cli/... — all packages pass except TestPathIsVersioned_ReftableRepo, which needs git ≥ 2.45 and fails identically on main here (git 2.43)
• gofmt -s, go vet, golangci-lint run ./cmd/entire/cli/... — no new issues (one pre-existing gosec finding in utils.go, untouched)
• go mod tidy — no changes; dupl at threshold 75 — no hits in the new files
• cargo fmt --check, cargo clippy --all-targets (0 warnings), cargo test (27 passed)
• cargo run --release --example bench — P 0.869 / R 0.270 / Spec 0.927
• recall/scripts/verify-offline.sh — 0 network syscalls
• Live: entire recall ingest on this repo → 125 checkpoints, coverage line PARTIAL · 5 of 125 checkpoints complete · 120 without transcript · …; queries return the expected commits as LEDGER hits with ◌ partial context: transcript unavailable where the transcript was unreadable
• mise run check end to end (mise is not installed on this machine; the individual steps above were run directly)
• Reviewer call on listed/audience for recall in agentHelpClassification — I took the safe default (unlisted, task-driven) per CLAUDE.md

Known limitations / follow-ups

• Recall 0.270: a high-precision screen, not a comprehensive detector.
• Thresholds need per-repository calibration outside this repo.
• Engrams carry no embedding vectors, so paraphrase reads as neutral rather than corroborated; populating semantic_vector with a local embedder is the highest-value next step.
• Prose with a slash in it (definition/search, owner/repo) can still trip the file check on long pasted prompts; per-sentence claims rather than per-turn is the fix.
• The graph index is per-HEAD; entire graph index before ingest avoids a ~90 s re-index.

🧙 Built with WOZCODE