Skip to content

multi: Register authenticated provisional lineage end to end#990

Open
ellemouton wants to merge 12 commits into
reorg-batchcanon-corefrom
reorg-lineage-producers
Open

multi: Register authenticated provisional lineage end to end#990
ellemouton wants to merge 12 commits into
reorg-batchcanon-corefrom
reorg-lineage-producers

Conversation

@ellemouton

@ellemouton ellemouton commented Jul 17, 2026

Copy link
Copy Markdown
Member

What this PR does

Connects the real VTXO creation and receipt paths to the batchcanon authority
introduced by #980. Without this PR the authority exists but production paths do
not consistently feed it.

It handles three production sources:

  1. Locally received round VTXOs.
  2. OOR-received VTXOs.
  3. Explicit custom VTXOs supplied as OOR inputs.

The indexer ancestry response supplies the serialized commitment transaction,
ordered commitment prevouts, CSV-delay evidence, and the parent lineage required
to validate a received VTXO. The recipient treats that response as untrusted
and authenticates: the commitment txid, the referenced output, every commitment
input, the input prevouts and their ordering, and the relationship between the
received VTXO and the supplied batch.

Registration happens before the VTXO is saved or exposed:

sequenceDiagram
    participant R as OOR receiver
    participant I as Indexer
    participant B as batchcanon
    participant V as VTXO store

    R->>I: Request ancestry
    I-->>R: Commitment, prevouts, and lineage
    R->>R: Authenticate evidence
    R->>B: Register complete lineage
    B-->>R: Registration durable
    R->>V: Materialize and save VTXO
Loading

Once registered, the client responds to later reorgs using its local chain
watches; it does not requery the indexer.

Wallet reporting uses the same authority: a lifecycle-live VTXO whose lineage is
reorged, conflicted, unseen, or still reconciling is excluded from spendable
balance and included in temporarily_unavailable_sat. Terminally invalidated
value is excluded from both buckets.

Review focus

  • Treat the indexer response as an untrusted input — are all bindings checked?
  • Are every TxIn and ordered prevout included and authenticated?
  • Is registration durably complete before exposure?
  • Are retries idempotent across every crash boundary?
  • Does legacy or incomplete evidence fail closed?

Full design & diagrams: REORG_SAFETY_PR_REVIEW_GUIDE.md §4.3. Overall
architecture and PR stack: lumos#454.

@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@ellemouton

Copy link
Copy Markdown
Member Author

@claude review this

@claude

claude Bot commented Jul 17, 2026

Copy link
Copy Markdown

Claude finished @ellemouton's task in 4m 34s —— View job


Review: PR #990 — Register authenticated provisional lineage end to end

I reviewed the full diff against origin/reorg-batchcanon-core. This is a well-structured, defensively-written PR. The trust boundary in batchcanon.EvidenceFromAncestryPaths is the centerpiece and it is thorough — it binds every server-supplied field (tx bytes → txid, tree output value/script → tx.TxOut[idx], each input outpoint/prevout → tx.TxIn[i]) rather than trusting caller assertions, and the fail-closed rollout story (legacy indexers → empty evidence → gate keeps lineage unusable) is coherent. Test coverage is good across all five producers.

Below are observations, ordered roughly by significance. None are blockers.


1. commitment_csv_expiry_delta is trusted, not authenticated

In batchcanon/rpc_evidence.go, every other evidence field is cross-checked against the serialized commitment tx, but CSVExpiryDelta (path.GetCommitmentCsvExpiryDelta()) is taken directly from the operator with only a > 0 check. A malicious operator can inflate/deflate it, which flows into Record.EffectiveExpiry() (ConfirmationHeight + CSVExpiryDelta). This is likely acceptable (the operator already controls tree construction, and CSV isn't derivable from the tx alone), but it's the one piece of "evidence" that isn't actually evidence. Worth a one-line comment at the trust boundary making that explicit so a future reader doesn't assume it's authenticated like the rest.

2. Asymmetric CSV validation between producer and receive paths

The receive/incoming paths enforce CSVExpiryDelta > 0 in three places (evidenceFromAncestryPath, encodeBatchEvidence/decodeBatchEvidence, validateIncomingEvidenceCoverage, registerIncomingBatchEvidence), but the round producer path in roundBatchRegistration (round/transitions.go) sets CSVExpiryDelta: int32(state.SweepDelay) with no positivity guard, and validateRegistration in batchcanon/manager.go does not check CSV either. SweepDelay is realistically always positive, so this is defensive-only, but given how consistently the receive side guards it, a matching guard (or a check inside validateRegistration) would close the gap and keep the invariant in one place. Fix this →

3. batchOutputIndex diverges from confirmationWatchScript on the out-of-range branch

batchOutputIndex (round/transitions.go) is documented as mirroring confirmationWatchScript so the persisted evidence watches the same output the chain backend does. They agree on the happy path, but when len(vtxoTrees) > 0 and the computed min index is out of range, confirmationWatchScript falls back to TxOut[0] while batchOutputIndex returns an error. So in that (shouldn't-happen) case the confirmation watch keys on output 0 while registration fails outright. The fail-closed direction is arguably safer, but the "mirrors confirmationWatchScript" comment slightly overstates the equivalence — either reconcile the fallback or note the intentional divergence.

4. Duplicated coverage-validation logic

oor.validateIncomingEvidenceCoverage (oor/incoming_batch_registration.go) and the inline coverage loop in vtxo.registerIncomingBatchEvidence (vtxo/incoming_batch_registration.go) are near-identical: per-item Validate(), CSV>0, "names commitment outside ancestry", duplicate detection, and the final len(seen) != len(ancestry) count. They live in different packages so a shared helper would need a home (perhaps batchcanon, alongside EvidenceFromAncestryPaths, taking the ancestry txid set). Not urgent, but this is exactly the kind of security-relevant check you don't want to drift between two copies.

5. Minor: emitReceived on registration failure

In vtxo/incoming_handler.go, a registration failure emits h.emitReceived(ctx, "failed") and returns an error. If the caller retries the receive, you can get failed then later received for the same transfer. Since RegisterBatch is idempotent this is only a metrics-cardinality nuance, not a correctness issue — flagging only for awareness.


Things I checked that are correct

  • OOR dependent-outpoint derivation is right. RegisterIncomingBatchEvidence builds the dependent as {Hash: chainhash.Hash(sessionID), Index: match.OutputIndex}. I confirmed SessionID is the Ark txid (see matchesIncomingVTXO comparing sessionID[:] to the candidate txid, and the events.go doc comments), and the materialized VTXO outpoint is {arkTxid, OutputIndex} — so these match. The vtxo path uses desc.Outpoint directly, which is the same value.
  • No double registration in OOR. Durable path registers in session_actor.driveOutboxEvents (outside the commit tx); the sync path registers in LocalPersistenceOutboxHandler gated on !hasActorDBTx(ctx). Mutually exclusive.
  • Registration-before-exposure ordering holds in round (before VTXOStore persist) and in vtxo (before save/notify), consistent with the fail-closed gate.
  • context.WithoutCancel in the round path correctly detaches the durable registration from request cancellation.
  • Proto changes (fields 6/7/8 on AncestryPath, new CommitmentInputEvidence) are purely additive with no field-number collisions.
  • Duplicate/conflict handling for same-commitment multi-leaf ancestry (equalBatchEvidence) is correct and tested.

Nice work — the authentication layer and the two-graph separation (actual inputs vs logical consumer edges) are clearly reasoned throughout.
· branch reorg-lineage-producers

@ellemouton

Copy link
Copy Markdown
Member Author

Review disposition for the current head: the CSV delay is intentionally operator-authored policy input because it is not derivable from the commitment transaction; all transaction-bound fields are authenticated, and the client still requires a positive delay. The output-index error path intentionally fails closed instead of copying the legacy confirmation-watch fallback. The duplicated receive-path coverage checks and failed-then-retried metric sequence are non-blocking cleanup notes; no correctness finding was left open in this PR.

@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from b1f9ab8 to 9655bde Compare July 17, 2026 23:27
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch 2 times, most recently from 8dbd209 to c566338 Compare July 17, 2026 23:38
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 9655bde to 659e6aa Compare July 17, 2026 23:38
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from c566338 to 899c934 Compare July 18, 2026 00:00
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 659e6aa to b2a5d3d Compare July 18, 2026 00:00
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from 899c934 to 9d3b6b0 Compare July 18, 2026 00:20
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from b2a5d3d to c121b30 Compare July 18, 2026 00:20
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from 9d3b6b0 to 095039e Compare July 18, 2026 00:38
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch 2 times, most recently from 49e2003 to 9d19e5d Compare July 20, 2026 16:10
@ellemouton

Copy link
Copy Markdown
Member Author

@claude review this — the current head adds canonicality-aware spendable and temporarily-unavailable balance reporting plus a real reorg/reconfirmation systest.

@claude

claude Bot commented Jul 20, 2026

Copy link
Copy Markdown

Claude finished @ellemouton's task in 2m 56s —— View job


Review: canonicality-aware balance reporting + reorg systest

Reviewed the head commits 3859c34 (proto fields) and 9d19e5d (classification + wiring + systest) against origin/reorg-batchcanon-core. This is a clean, well-tested increment. The core idea — a third balance bucket for reversible lineage limbo that is carved out of both vtxo_balance_sat and total_confirmed_sat — is implemented consistently end to end. No blockers.

What's correct

  • Spendable classification mirrors the coin-selection gate. ClassifyCanonicalityBalance (vtxo/filter.go:130) counts a VTXO as spendable iff availability.Usable(), which is exactly the predicate gateUnavailableLineage uses to admit candidates. So the reported spendable figure can't drift from what coin selection will actually let you spend — the invariant you'd most want held here.
  • No double counting. The three live buckets are disjoint by construction: ClassifyCanonicalityBalance and SumPendingBalance both key off Status, and only SumPendingBalance touches PendingForfeit/Forfeiting/Spending while classification only touches VTXOStatusLive. Exiting VTXOs come from a separate ListVTXOsByStatusLight query. Nothing overlaps.
  • total_confirmed_sat correctly excludes limbo. rpc_server.go:868 sums BoardingConfirmedSat + VtxoBalanceSat (now the spendable-only figure), matching both proto docs.
  • Terminal Invalidated contributes to neither bucket — right call; it's no longer wallet liquidity, and the case availability != batchcanon.Invalidated guard keeps it out of the temporary bucket.
  • Worst-parent semantics flow through LineageCommitmentTxIDsLineageAvailabilityCombineAvailability, and the multi-parent test (TestCanonicalityAwareVTXOBalanceUsesWorstParent) pins it: one reorged parent drags the whole VTXO into limbo.
  • Fail-closed error propagation — a broken reader surfaces an RPC Internal error rather than a misleading zero/spendable figure (TestCanonicalityAwareVTXOBalancePropagatesReaderError), consistent with the strictness rationale already documented above the boarding/VTXO fetches.
  • Nil-reader legacy path is behavior-neutral (SumSpendableBalance), and the systest asserts the real durable store across a live reorg → reconfirm cycle at all three states (Provisional → ReorgedOut → reconfirmed Provisional), using the same ListLiveVTXOs + ClassifyCanonicalityBalance path the RPC uses.
  • Proto changes are purely additive (field 6 on BalanceResponse, field 11 on GetBalanceResponse), no field-number collisions; generated .pb.go was regenerated, not hand-edited.

Observations (all non-blocking)

1. Proto docs omit the unseen case that the bucket actually includes. The temporary bucket is everything that is !Usable() && != Invalidated, which per the test (unseenBatch → 50k in unavailable) and the commit message ("reorged, conflicted, unseen, and reconciling") deliberately includes AvailabilityUnknown (unseen lineage) and LineageReconciling. But the field docs enumerate a narrower set:

  • wallet.proto:955 — "while its batch lineage is in reorg/conflict limbo or is being reconciled"
  • daemon.proto:564 — "This includes reorg/conflict limbo and lineage reconciliation"

Neither mentions unseen/unregistered lineage. Since a client rendering this figure will see freshly-created-but-unconfirmed live VTXOs land here, it's worth a word in the doc so the wire contract matches the behavior the test locks in. Fix this →

2. UX consideration for the unseen window. Following from (1): a just-received round/OOR VTXO whose commitment isn't confirmed yet is Live but AvailabilityUnknown, so it will briefly appear as temporarily_unavailable rather than as inbound-pending. That's the fail-closed-correct answer (it truly isn't spendable yet), but "temporarily unavailable" reads more alarming than "pending" for a normal first-confirmation wait. Worth a conscious confirmation that this is the intended surface, since it's the one state here that isn't actually a fault condition.

3. Minor nil-guard asymmetry. The non-nil path guards desc == nil (filter.go:117), but the nil-reader path delegates to SumSpendableBalanceFilterDescriptors, which dereferences d.Status without a nil check and would panic on a nil element. ListLiveVTXOs never yields nil entries so this isn't reachable today; flagging only because the two paths now differ on an input the new one explicitly defends against.

4. Per-RPC store fan-out. GetBalance now issues one GetBatch per (live VTXO × distinct parent) via LineageAvailability. This matches the gate's existing per-candidate pattern and is fine at current scales, but it does turn a previously in-memory sum into O(N·P) store reads on a hot read RPC. Not worth changing now — noting for when wallet VTXO counts grow.

Nice, tight change — the classification predicate reusing Usable() is exactly the right way to keep the balance figure and the admission gate from ever disagreeing.
· branch reorg-lineage-producers

@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 9d19e5d to 97bd459 Compare July 20, 2026 16:24
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from 095039e to 7f636e7 Compare July 20, 2026 16:24
@ellemouton

Copy link
Copy Markdown
Member Author

@claude review this — final head is now rebased onto current main and includes canonicality-aware temporarily-unavailable balance reporting plus the real reorg systest.

@claude

claude Bot commented Jul 20, 2026

Copy link
Copy Markdown

Claude finished @ellemouton's task in 2m 8s —— View job


Review: PR #990 — final head (rebased onto main)

  • Read diff vs origin/reorg-batchcanon-core
  • Review evidence authentication (batchcanon)
  • Review producer wiring (round, oor, vtxo)
  • Review balance classification + RPC wiring (vtxo/filter.go, waved)
  • Review reorg systest
  • Post review

Re-reviewed the full stack (7 commits) against origin/reorg-batchcanon-core after the rebase onto current main. The rebase is clean — no regressions in the producer wiring or the two-graph separation that the two prior passes signed off on, and the balance-reporting increment is now folded in coherently. No blockers. Everything flagged below is non-blocking, and most of it is a status check on prior observations rather than anything new.

Rebase / correctness confirmations

  • Balance classification still mirrors the coin-selection gate. ClassifyCanonicalityBalance (vtxo/filter.go:130) counts value as spendable iff availability.Usable() — the same predicate coin selection admits on — and buckets !Usable() && != Invalidated as temporarily unavailable. The reported figure can't drift from what a spend will actually let you use. Terminal Invalidated contributes to neither bucket. ✅
  • total_confirmed_sat excludes limbo. GetBalance now sums the spendable-only VtxoBalanceSat (rpc_server.go), so both the new vtxo_temporarily_unavailable_sat (daemon field 11) and temporarily_unavailable_sat (wallet field 6) are carved out of the confirmed total. Proto changes are purely additive. ✅
  • Explicit-OOR-input gate is symmetric with wallet coin selection. requireCustomInputLineage (rpc_server.go) runs batchcanon.LineageBlocked over LineageCommitmentTxIDs(desc) before checkpoint signing, mapping Invalidated → FailedPrecondition and reorg/reconcile limbo → Unavailable, and no-ops when batchCanonStore is nil (legacy). Closes the bypass where a reorged-out VTXO could reach signing through the custom-input path. ✅
  • Registration-before-exposure ordering still holds: round registers inside InputSigSentState.ProcessEvent before the VTXOStore persist, under context.WithoutCancel, and fails the round at its pre-exposure checkpoint on registration error (round/transitions.go). oor/vtxo register before materialize/save. Fail-closed direction is preserved throughout. ✅
  • New systest exercises the real durable store. assertCanonicalityBalance (systest/batch_canonicality_gate_test.go) drives ListLiveVTXOs + ClassifyCanonicalityBalance against the real batchcanon.Reader across Provisional → ReorgedOut → reconfirmed-Provisional, asserting the exact spendable/unavailable split at each state — the same path GetBalance uses. ✅

Non-blocking observations (status of prior notes)

1. Proto docs still omit the "unseen" bucket. ClassifyCanonicalityBalance puts AvailabilityUnknown (a freshly-created, not-yet-confirmed live VTXO) into the temporary bucket, but wallet.proto:952 ("reorg/conflict limbo or is being reconciled") and daemon.proto:562 ("reorg/conflict limbo and lineage reconciliation") don't mention unseen/unregistered lineage. A client will briefly render a normal first-confirmation wait as temporarily_unavailable; worth one word so the wire contract matches the behavior the systest locks in. (Carried over from the balance-PR review; still open.) Fix this →

2. CSVExpiryDelta positivity is still only guarded on the receive path. The receive paths reject CSVExpiryDelta <= 0 (batchcanon/rpc_evidence.go:176, oor/incoming_batch_registration.go:116), but the round producer sets CSVExpiryDelta: int32(state.SweepDelay) (round/transitions.go) with no guard, and validateRegistration (batchcanon/manager.go:305) checks every field except CSV. So a zero/negative SweepDelay would register and make EffectiveExpiry() collapse to ConfirmationHeight. SweepDelay is realistically always positive — this is defensive-only, and you've already dispositioned it as intentional. Flagging only that the rebase didn't change it; a one-line CSVExpiryDelta > 0 check in validateRegistration would keep the invariant in one place for both producers.

3. Minor nil-guard asymmetry. The non-nil path guards desc == nil (vtxo/filter.go:117), but the nil-reader path delegates to SumSpendableBalance → FilterDescriptors, which dereferences d.Status without a nil check. ListLiveVTXOs never yields nil entries, so unreachable today — noting only because the two paths now differ on an input the new one explicitly defends against.

Note on validation

I could not run go build / go test in this environment (build command not permitted here), so the confirmations above are from code reading against the diff. The PR body reports make fmt-changed, make lint-changed-local, make tidy-module-check, go test ./vtxo ./waved ./oor ./round, and F2/F3/F6/F7 real-chain systests all passing.

The authentication layer and the reuse of Usable() as the single admission-and-balance predicate remain the strongest parts of this change — balance and coin selection are structurally incapable of disagreeing.
· branch reorg-lineage-producers

@ellemouton

Copy link
Copy Markdown
Member Author

@claude review this — addressed the two actionable non-blocking notes: temporary-balance docs now include unseen/unregistered lineage, and all batch registrations now enforce positive CSV expiry with regression coverage.

@ellemouton

Copy link
Copy Markdown
Member Author

Review disposition: temporary-balance wire docs and positive CSV registration validation are fixed in the current head. The nil-reader descriptor note is unreachable because ListLiveVTXOs does not return nil descriptors; no correctness finding is left open.

@claude

claude Bot commented Jul 20, 2026

Copy link
Copy Markdown

Claude finished @ellemouton's task in 2m 55s —— View job


Review: follow-up fixes (CSV positivity + temporary-balance docs)

  • Review CSV-expiry positivity commit (c4cd79b)
  • Review temporary-balance docs commit (37ab56c)
  • Confirm regression coverage
  • Post review

Reviewed the two new commits on top of the previously-signed-off stack. Both notes are addressed correctly. One small formatting issue that will fail CI.

✅ CSV-expiry positivity (c4cd79b) — correctly centralized

Putting the req.CSVExpiryDelta <= 0 check in validateRegistration (batchcanon/manager.go:317) is the right fix, and better than the per-producer guard I suggested. validateRegistration is the single chokepoint every registration flows through — both the actor path (handleRegisterBatch, manager.go:225) and the store path (record.go:65) — so this now covers the round producer (SweepDelay), the receive/OOR import paths, and any future producer with one invariant, exactly the "keep it in one place" outcome. EffectiveExpiry() can no longer collapse to ConfirmationHeight from a zero/negative delta. Regression coverage is good: the new "non-positive CSV expiry delta" table case in registration_validation_test.go:39 asserts the rejection, and the two harness defaults (manager_test.go, validRegistrationRequest) keep existing fixtures valid.

✅ Temporary-balance docs (37ab56c) — matches behavior

Both field docs now read "unseen, unregistered, in reorg/conflict limbo, or being reconciled" (wallet.proto:954, daemon.proto:562), which correctly describes what ClassifyCanonicalityBalance actually buckets (AvailabilityUnknown / LineageReconciling → temporary). The wire contract now matches the behavior the systest locks in, closing the doc gap from the prior two reviews. .pb.go was regenerated (not hand-edited), and proto and generated comments agree.


🔴 One actionable: registration_validation_test.go:138 is not gofmt-clean

The new field breaks the struct's alignment block:

BatchOutputIndex:     1,
ConfirmationPkScript: watchScript,
CSVExpiryDelta:        144,   // <- one extra space; value is shifted a column right

ConfirmationPkScript is the longest key in the group, so gofmt wants 7 spaces after CSVExpiryDelta: (aligning 144 under watchScript), but there are 8. This will fail make fmt-changed-check / the lint gate in CI. A make fmt-changed pass fixes it. Fix this →

(I couldn't run make fmt-changed-check directly in this environment — permission-gated — but the misalignment is visible in the source: 144 sits one column right of watchScript above it.)


Status of remaining prior notes (all non-blocking, unchanged)

  • Nil-guard asymmetry (vtxo/filter.go:117 guards desc == nil; the nil-reader path via SumSpendableBalance → FilterDescriptors does not) — still present, still unreachable since ListLiveVTXOs yields no nil entries. Informational only.
  • Per-RPC store fan-out in GetBalance (O(N·P) GetBatch reads) — unchanged, fine at current scales.

Everything else from the three prior passes still holds. Nice, focused follow-up — the CSV fix in particular lands the invariant in the cleanest possible spot.
· branch reorg-lineage-producers

@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from 37ab56c to 6c63040 Compare July 20, 2026 16:49
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from 7f636e7 to 8943a38 Compare July 20, 2026 18:46
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from d7cb561 to c6fc5e3 Compare July 20, 2026 18:46
@ellemouton

Copy link
Copy Markdown
Member Author

@claude review this — current restacked head c6fc5e36 after rebasing the full stack onto the latest upstream base. Please verify prior findings remain addressed and flag any new blocking issues.

@claude

claude Bot commented Jul 20, 2026

Copy link
Copy Markdown

Claude finished @ellemouton's task in 1m 20s —— View job


Review: PR #990 — restacked head c6fc5e36

  • Diff current head vs origin/reorg-batchcanon-core; confirm restack is clean
  • Verify prior findings remain addressed (CSV positivity, temp-balance docs, sweep-delay durability)
  • Re-check evidence authentication / producer wiring / balance classification
  • Scan for any new blocking issues introduced by the rebase
  • Post review

Re-checked the current head (10 commits, tip c6fc5e36) against origin/reorg-batchcanon-core (now at 8943a38 batchcanon: Serialize startup reconciliation). The commit set and subjects are identical to the head I signed off across the prior passes — this is a clean restack onto the newer base, with no content drift, no conflict markers in batchcanon/round/oor/vtxo/waved, and every previously-verified invariant intact. No blockers, nothing new to flag.

Restack integrity — prior findings all still addressed

  • CSV positivity is centralized. validateRegistration still rejects CSVExpiryDelta <= 0 at the single chokepoint (batchcanon/manager.go:317), covering the round producer (SweepDelay) and the receive/OOR import paths. The receive-path guard (batchcanon/rpc_evidence.go:176) is also intact. EffectiveExpiry() can't collapse to ConfirmationHeight. ✅
  • Sweep-delay durability round-trip survives the rebase. Write stamps SweepDelay: int32(inputSigState.SweepDelay) (db/round_store.go:335), read restores uint32(dbRound.SweepDelay) (db/round_store.go:1242); LatestMigrationVersion is still 17. ✅
  • Temporary-balance docs still name the unseen/unregistered bucket matching ClassifyCanonicalityBalance (proto edits carried through). ✅
  • Balance classification mirrors the coin-selection gate. ClassifyCanonicalityBalance (vtxo/filter.go:130) counts value as spendable iff availability.Usable(), buckets !Usable() && != Invalidated as temporarily unavailable, and terminal Invalidated contributes to neither. The desc == nil guard (filter.go:117) is present. Balance and admission can't disagree. ✅
  • total_confirmed_sat excludes limbo — still BoardingConfirmedSat + VtxoBalanceSat (spendable-only) at waved/rpc_server.go:868. ✅
  • Evidence-authentication centerpiece is unchanged. EvidenceFromAncestryPaths still binds every server-supplied field to the serialized commitment tx — output index bounds-check against len(tx.TxOut) (rpc_evidence.go:108), input-count equality (:126), and per-input outpoint match against tx.TxIn[i].PreviousOutPoint (:145). ✅
  • Producer wiring / registration-before-exposure holds: round registers under context.WithoutCancel before the VTXOStore persist (round/transitions.go:4730), and the two-graph separation (actual commitment inputs vs logical consumer edges) is preserved. oor/vtxo register before materialize/save. ✅
  • Explicit-OOR-input gate requireCustomInputLineage (rpc_server.go:296) still runs batchcanon.LineageBlocked before checkpoint signing (:3131). ✅

Status of standing non-blocking notes (all previously dispositioned — unchanged)

  • Migration-window straddle — a round already in input_sig_sent at upgrade through 000017 back-fills sweep_delay = 0 and holds fail-closed at its pre-exposure checkpoint post-restart; one in-flight round per upgrade, intentional-lossy default.
  • uint32int32 narrowing on SweepDelay — lossless for any realistic CSV.
  • CSV-delta as operator policy (authenticated tx fields vs operator-authored delta), the fail-closed output-index error path, the nil-reader descriptor guard, and the per-RPC GetBalance store fan-out (O(N·P), fine at current scale) — all as you dispositioned.

Note on validation

go build/go test/gofmt are permission-gated in this environment, so the above is from reading the diff and tracing call paths against the newer base, not from re-running the toolchain. The PR body reports make fmt-changed, make lint-changed-local, make tidy-module-check, go test ./vtxo ./waved ./oor ./round, and F2/F3/F6/F7 real-chain systests green.

The restack is clean — the authentication layer and the reuse of Usable() as the single admission-and-balance predicate remain intact and untouched by the rebase.
· branch reorg-lineage-producers

@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from c6fc5e3 to f3cf7bd Compare July 20, 2026 20:05
@ellemouton

Copy link
Copy Markdown
Member Author

@claude review this

@claude

claude Bot commented Jul 20, 2026

Copy link
Copy Markdown

Claude finished @ellemouton's task in 2m 13s —— View job


Review: PR #990 — new head f3cf7bd (anchor watches before confirmation)

  • Diff current head vs origin/reorg-batchcanon-core; identify what's new
  • Review the new batchcanon: Anchor watches before confirmation commit
  • Confirm prior findings remain addressed
  • Post review

Only one commit is new since the last signed-off head (c6fc5e36): f3cf7bd batchcanon: Anchor watches before confirmation. Everything below is scoped to it — the rest of the stack (evidence authentication, producer wiring, balance classification, sweep-delay durability) is unchanged and already reviewed across the prior passes. No blockers.

This is a tight, correct fix for a real gap: without a height hint, a light-client watch installed after the commitment already confirmed would scan forward from "now" and never see the already-mined confirmation, leaving the VTXO stuck Unseen. Threading a pre-confirmation height through registration closes that.

What's correct

  • The hint is plumbed end to end and actually consumed. WatchHeightHint flows BatchEvidenceRegisterRequest (record.go:58) → Record.WatchHeightHint (manager.go:257) → watchFromRecord heightHint (manager.go:1427), so it reaches the real watch rather than being a dead field. It's persisted on Record and survives restart, matching the doc's "retained across restarts so delayed watch installation cannot miss already-mined evidence."
  • The two producers pick the right anchor for their situation.
    • Round path uses env.StartHeight (transitions.go:4734) — the round's creation height, captured before broadcast/signing, so it's a conservative lower bound that is robust regardless of when the confirmation lands. StartHeight is durable (restored from round.StartHeight on resume, actor.go:823), so the anchor is stable across restart. The new assertion registration.WatchHeightHint == h.env.StartHeight (batch_registration_test.go:77) pins it.
    • Indexer/receive path uses the observed commitment_height (rpc_evidence.go:177), which is exactly right for a delayed/replayed receive where the commitment is already mined — scanning from the commitment height catches the past confirmation, whereas anchoring at the local current height would step over it.
  • Negative-height rejection is correct and tested. commitment_height is int32 (indexer.pb.go:1500); the < 0 guard (rpc_evidence.go:76) rejects a nonsensical value before the uint32 conversion (which would otherwise wrap to a huge height and guarantee the watch misses everything). The "negative commitment height" table case (rpc_evidence_test.go:80) covers it.
  • Backward-compatible durable encoding. The new OOR TLV record uses type 13 (actor_durable_message.go:150) — odd, i.e. optional — so an OOR durable message persisted before this commit decodes cleanly with watchHeight left at its 0 zero value. A 0 hint scans from genesis: more work, but fail-safe (over-scans, never under-scans). New evidence carries the real hint. Continues the existing 7/9/11 odd-type sequence.
  • equalBatchEvidence correctly includes the new field (rpc_evidence.go:214). Two ancestry leaves for the same commitment carry the same commitment_height (same tx, same conf height), so this can't manufacture spurious conflicts in the multi-leaf dedup path.

Non-blocking observations

1. commitment_height is operator-authored, not tx-authenticated — same class as CSVExpiryDelta. Confirmation height isn't derivable from the commitment tx, so the receive path takes it on the operator's word (only the < 0 check gates it). A malicious operator supplying a height above the true confirmation would make the light-client watch scan past the real confirmation and leave the received VTXO stuck Unseen. That is a liveness-only, fail-closed outcome (the VTXO never becomes spendable; it never becomes spuriously spendable), and it's strictly weaker than what a hostile operator can already do by withholding evidence entirely. So this is consistent with your existing "CSV delta is operator policy, not evidence" disposition — worth the same one-line acknowledgement at the trust boundary that WatchHeightHint from the indexer is trusted policy input, unlike the tx-bound fields around it. Note the round producer path has no such dependence: StartHeight is locally sourced, so the producer side is self-protecting; only the receive side trusts the operator for this liveness property.

2. Round-path robustness rests on StartHeight being a true pre-confirmation lower bound. It is today (round creation strictly precedes broadcast), and using the round's own start height rather than a live chain read is the right call for restart determinism. Flagging only that the invariant "StartHeight ≤ actual commitment confirmation height" is the load-bearing assumption for the round producer's watch never missing its confirmation — safe as written.

Status of standing prior notes (all previously dispositioned, unchanged by this commit)

  • CSV positivity centralized in validateRegistration; sweep-delay durability round-trip (db migration 000017); temporary-balance wire docs naming the unseen/unregistered bucket; Usable() as the single admission-and-balance predicate — all intact.
  • Migration-window straddle (sweep_delay = 0 back-fill), uint32int32 narrowing, nil-reader descriptor guard, per-RPC GetBalance fan-out — all as you dispositioned.

Note on validation

go build/go test/gofmt are permission-gated in this environment, so the above is from reading the diff and tracing the call paths, not from re-running the toolchain. The PR body reports make fmt-changed, make lint-changed-local, make tidy-module-check, go test ./vtxo ./waved ./oor ./round, and F2/F3/F6/F7 real-chain systests green.

Clean, well-scoped commit — the split between a locally-sourced conservative anchor (round) and the observed confirmation height (delayed receive) is the right shape, and the odd-TLV / 0-hint fallback keeps it safe for both old persisted messages and legacy indexers.
· branch reorg-lineage-producers

@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from f3cf7bd to ecfc4c2 Compare July 20, 2026 20:18
@ellemouton

Copy link
Copy Markdown
Member Author

@claude review this — current head ecfc4c22 is the final restack on client core 4991fb72; local rounds and durable indexer/OOR evidence now supply a pre-confirmation scan height.

@claude

claude Bot commented Jul 20, 2026

Copy link
Copy Markdown

Claude finished @ellemouton's task in 1m 35s —— View job


Review: PR #990 — restacked head ecfc4c22 (final restack on client core 4991fb72)

  • Diff current head vs origin/reorg-batchcanon-core; confirm restack is clean
  • Verify prior findings remain addressed
  • Scan for new blocking issues from the rebase
  • Post review

Re-checked the current head (11 commits, tip ecfc4c22) against origin/reorg-batchcanon-core, which has advanced to 4991fb72 batchcanon: Retain conservative watch hint. This is a clean restack of the same 11-commit set onto the newer client core — no conflict markers in batchcanon/round/oor/vtxo/waved, and every previously-verified invariant intact. No blockers, nothing new to flag.

The one meaningful delta from the last pass — and it's an improvement

The new base absorbs the consumer half of the watch-height work: WatchHeightHint on RegisterBatchRequest/Record and its consumption at batchcanon/manager.go:257 and :1424 (→ heightHint → the real watch at :453/:511/:1427) now live in the base. So the batchcanon: Anchor watches before confirmation commit (ecfc4c2) is now correctly scoped down to just the producer wiring — adding WatchHeightHint to BatchEvidence and sourcing it — which is exactly the right seam for this PR ("register authenticated provisional lineage") versus the client-core base. I confirmed the field is actually consumed end to end, not a dead field.

Restack integrity — anchor-watch producer wiring intact

  • Round producer uses the local pre-confirmation anchor. roundBatchRegistration now takes watchHeightHint uint32 (round/transitions.go:4406) and ProcessEvent passes env.StartHeight (:4734). StartHeight is durable (restored from round.StartHeight on resume, round/actor.go:820-823), so the anchor is a conservative, restart-stable lower bound (StartHeight ≤ actual confirmation height). ✅
  • Indexer/receive path uses the observed commitment height. evidenceFromAncestryPath rejects commitment_height < 0 before the uint32 conversion (rpc_evidence.go:76) and sets WatchHeightHint: uint32(path.GetCommitmentHeight()) (:177) — right for a delayed/replayed receive where the commitment is already mined. ✅
  • OOR durable persistence carries the hint on odd TLV type 13 (oor/actor_durable_message.go:150), so pre-upgrade persisted messages decode with watchHeight == 0 (scans from genesis: over-scans, never under-scans), and new evidence carries the real hint. Encode/decode round-trip present (:701, :749, :785). ✅
  • equalBatchEvidence includes WatchHeightHint (rpc_evidence.go:214); same-commitment multi-leaf ancestry carries the same height, so no spurious dedup conflicts. ✅

Prior findings — all still addressed after the restack

  • CSV positivity centralized. validateRegistration still rejects CSVExpiryDelta <= 0 at the single chokepoint (batchcanon/manager.go:319), covering the round producer (SweepDelay) and receive/OOR import. ✅
  • Sweep-delay durability round-trip intact (db/round_store.go; migration 000017, LatestMigrationVersion 17). ✅
  • Temporary-balance wire docs name the unseen/unregistered bucket, matching ClassifyCanonicalityBalance (vtxo/filter.go). ✅
  • Balance classification mirrors the coin-selection gate (Usable() as the single admission-and-balance predicate); total_confirmed_sat excludes limbo. ✅
  • Evidence authentication centerpiece unchanged — every server-supplied field bound to the serialized commitment tx (output-index bounds, input-count equality, per-input outpoint match). ✅
  • Registration-before-exposure ordering and the two-graph separation (actual commitment inputs vs logical consumer edges) preserved. ✅

Standing non-blocking notes (all previously dispositioned — unchanged)

  • WatchHeightHint/CSVExpiryDelta from the indexer are operator-authored policy, not tx-authenticated — both are liveness-only, fail-closed on a hostile value (VTXO stuck Unseen/expiry never collapses spuriously); the round producer is self-protecting (locally-sourced StartHeight). Consistent with your existing disposition.
  • Migration-window straddle (sweep_delay = 0 back-fill → fail-closed hold), uint32int32 narrowing, nil-reader descriptor guard (unreachable), and per-RPC GetBalance fan-out (O(N·P), fine at current scale) — all as dispositioned.

Note on validation

go build/go test/gofmt are permission-gated in this environment, so the above is from reading the diff and tracing call paths against the newer base, not from re-running the toolchain. The PR body reports make fmt-changed, make lint-changed-local, make tidy-module-check, go test ./vtxo ./waved ./oor ./round, and F2/F3/F6/F7 real-chain systests green.

The restack is clean, and moving the watch-hint consumer into the base while this PR keeps only the producer wiring is the cleaner layering — the authentication layer and the Usable() single-predicate design remain intact and untouched.
· branch reorg-lineage-producers

Build complete batch evidence from the retained commitment PSBT when a
round confirms. Await durable canonicality registration before saving
any new VTXO so the fail-closed admission gate never observes
unregistered liquidity.

Refresh registrations bind each consumed VTXO to its next lifecycle
revision and complete creator lineage, allowing a terminally replaced
round to restore only the exact forfeiture it owns.
Carry the commitment transaction and every input prevout in ancestry
responses. Validate the transaction, output, and input bindings before
receive paths register canonicality watches.
Persist authenticated evidence in the durable receive event and await
canonicality registration before staging incoming VTXOs. Legacy indexers
may omit the full additive extension, while partial or conflicting
evidence fails closed.
Carry authenticated evidence through the round-receive ancestry lookup.
Await canonicality registration before saving or notifying the VTXO
manager. Missing legacy evidence remains blocked by the gate.
Custom OOR inputs bypass wallet coin selection and its fail-closed
lineage gate. Reuse the full-lineage reader before maturity checks so
reorged or invalidated inputs never reach checkpoint signing.
Expose reversible lineage limbo separately from spendable and pending
VTXO balances so clients can explain why value is temporarily unusable.
Keep reorged, conflicted, unseen, and reconciling lineage out of
the spendable figure while preserving it in a temporary-unavailability
bucket. Exercise the classification across a real reorg and
reconfirmation.
Reject malformed local or imported registration evidence before it can
produce a height-derived expiry at the confirmation block itself.
Include unseen and unregistered lineage in the balance field contract so
clients can explain the same fail-closed bucket the daemon reports.
Batch registration after a restart must use the immutable CSV delay
under which the client signed the round. Persist and restore that
per-round term instead of reconstructing it as zero.
Pass the local round start height and indexed commitment height into
batch canonicality registration. Persist the indexer hint through OOR
durable messages so delayed and replayed receive paths scan across
already-mined confirmations.
@ellemouton
ellemouton force-pushed the reorg-batchcanon-core branch from a4f2c42 to f2bfa75 Compare July 20, 2026 22:08
@ellemouton
ellemouton force-pushed the reorg-lineage-producers branch from a08dd0b to a7d456b Compare July 20, 2026 22:08
@ellemouton

Copy link
Copy Markdown
Member Author

@claude review this — head was rebuilt (height-hint watch clamp squashed into the batchcanon watch-hint story; client re-pins). CI is green. Please re-review the current head.

@claude

claude Bot commented Jul 21, 2026

Copy link
Copy Markdown

Claude finished @ellemouton's task in 2m 12s —— View job


Review: PR #990 — rebuilt head a7d456b (watch-clamp squashed into the watch-hint story)

  • Diff current head vs origin/reorg-batchcanon-core; identify what changed
  • Review the squashed height-hint watch-clamp change
  • Confirm prior findings remain addressed
  • Post review

Re-checked the current head (11 commits, tip a7d456b) against origin/reorg-batchcanon-core, which has advanced to f2bfa75 batchcanon: Retain conservative watch hint. The commit set and subjects are unchanged from the head I signed off previously — this is a rebuild where the watch-clamp work moved into the base and this PR's batchcanon: Anchor watches before confirmation (a7d456b) keeps only the producer wiring. Diff of that commit confirms it is producer-only (feed WatchHeightHint, persist it through OOR TLV, reject negative indexer height). No blockers, nothing new to flag.

The rebuild is a clean layering improvement — and it closes a prior note

The base commit f2bfa75 now owns the consumer half of the watch-hint contract, and it's worth noting it directly resolves the round-path robustness caveat I raised last pass:

  • Clamp floor lives in the base, at the watch seam. minWatchHeightHint = 1 (batchcanon/manager.go:42), applied in armWatches (:466-468) and threaded into the spend watch (:527, :545-552). A persisted hint of 0 — legitimately produced by the round path when env.StartHeight is genesis on a fresh chain — would otherwise be rejected by the notifier ("a height hint greater than 0 must be provided") and leave a fail-closed batch permanently unwatched and unusable. Clamping up only widens the scan window, so it never steps over the confirmation the watch exists to observe. This means the round producer feeding env.StartHeight (which can be 0) is now safe by construction, not just "safe because StartHeight is realistically > 0." ✅
  • Repeated-registration height drift is retained, not quarantined. The base allows a later registration (e.g. a reconfirmation) to report a different observation height while keeping the original lower scan point — safe (over-scans) and avoids spuriously conflicting otherwise-identical evidence.

Producer-wiring commit (a7d456b) — correct

  • Field is plumbed and consumed end to end. BatchEvidence.WatchHeightHintRegisterRequest (record.go:58) → Record.WatchHeightHint (manager.go:268) → batchWatch.heightHint (manager.go:1443) → clamped heightHint at arm time. Not a dead field. ✅
  • Round producer uses the local pre-confirmation anchor. roundBatchRegistration takes watchHeightHint uint32 (round/transitions.go:4405) and ProcessEvent passes env.StartHeight (:4734) — a conservative, restart-stable lower bound (StartHeight restored on resume). Test pins registration.WatchHeightHint == h.env.StartHeight (round/batch_registration_test.go:77). ✅
  • Indexer/receive path uses the observed commitment height, rejecting commitment_height < 0 before the uint32 conversion (rpc_evidence.go:76-77) so a nonsensical value can't wrap to a huge height that guarantees the watch misses everything. "negative commitment height" table case added (rpc_evidence_test.go:80). ✅
  • OOR durable persistence carries the hint on odd TLV type 13 (oor/actor_durable_message.go:150), so pre-upgrade persisted messages decode with watchHeight == 0 (→ clamped to floor, over-scans, never under-scans) and new evidence carries the real hint. Encode/decode round-trip present. ✅
  • equalBatchEvidence includes WatchHeightHint (rpc_evidence.go:214). Same-commitment multi-leaf ancestry shares one commitment height (same tx), so this can't manufacture spurious dedup conflicts. ✅

Prior findings — all still addressed

  • CSV positivity centralized in validateRegistration (batchcanon/manager.go:330); WatchHeightHint intentionally needs no positivity guard here since the base clamps it at arm time. ✅
  • Sweep-delay durability round-trip intact (db/round_store.go; migration 000017). ✅
  • Temporary-balance wire docs name the unseen/unregistered bucket, matching ClassifyCanonicalityBalance. ✅
  • Balance classification mirrors the coin-selection gate via Usable(); total_confirmed_sat excludes limbo. ✅
  • Evidence-authentication centerpiece unchanged — every server-supplied field bound to the serialized commitment tx (output-index bounds, input-count equality, per-input outpoint match). ✅
  • Registration-before-exposure ordering and the two-graph separation preserved. ✅

Standing non-blocking notes (all previously dispositioned — unchanged)

  • WatchHeightHint / CSVExpiryDelta from the indexer are operator-authored policy, not tx-authenticated. Both are liveness-only, fail-closed on a hostile value (VTXO stuck Unseen / expiry never collapses spuriously); the round producer is self-protecting (locally-sourced StartHeight). With the clamp now in place, a hostile high commitment height is still only a liveness downgrade, never a safety one. Consistent with your existing disposition.
  • Migration-window straddle (sweep_delay = 0 back-fill → fail-closed hold), uint32int32 narrowing, nil-reader descriptor guard (unreachable), and per-RPC GetBalance fan-out (O(N·P), fine at current scale) — all as dispositioned.

Note on validation

go build/go test/gofmt are permission-gated in this environment, so the above is from reading the diff and tracing call paths against the newer base, not from re-running the toolchain. You report CI green on this exact head, consistent with what I can see.

The rebuild is clean: moving the clamp+retention consumer into the base while this PR keeps only the producer wiring is the right seam, and the clamp closes the last "safe-in-practice-only" edge on the round path. The authentication layer and the Usable() single-predicate design remain intact.
· branch reorg-lineage-producers

The wired VTXO admission gate reads lineage availability straight from
the durable store via LineageBlocked. When a reorg/conflict observation
could not be persisted (a transient ApplyObservation failure), the
in-memory watch was marked not-ready but the stale durable row stayed
Ready(), so the gate kept admitting a VTXO whose commitment had just
left the best chain. The overlay the code comments promised only ever
protected the unwired actor QueryLineage path.

Make the manager itself a batchcanon.Reader whose GetBatch overlays the
in-memory watch: a not-ready watch forces the returned record not-ready,
even when the durable row still reads Ready. Thread the manager (not the
raw store) into the gate. The overlay only downgrades ready->not-ready,
so it is strictly fail-closed. Add a wired-path test proving admission
closes after a failed reorg persist.

@Roasbeef Roasbeef left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this review, I found two P1 admission gaps and two recovery/evidence issues at the current head. Registration-before-exposure and the fail-closed overlay are the right direction, but the direct reader still has a stale-read interleaving, and unroll/fraud doesn't consume the new gate. I'm requesting changes on those paths before we use this PR as the client half of the #454 capstone.

Comment thread batchcanon/manager.go
func (m *Manager) GetBatch(ctx context.Context, txid chainhash.Hash) (*Record,
error) {

record, err := m.cfg.Store.GetBatch(ctx, txid)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] We need to take m.mu before the store read here. Right now the read can return Provisional@r, then a reorg callback takes the mutex and successfully persists ReorgedOut@(r+1). That complete snapshot leaves w.ready=true, so the overlay below doesn't downgrade the stale record and the caller still admits it. I think the clean fix is to linearize the store read with the watch snapshot under the same lock, then add a deterministic interleaving test.

Comment thread vtxo/manager.go
// activated once the batch producers (round, OOR) register their
// batches, or every unregistered VTXO is excluded.
BatchCanonicality batchcanon.Store
BatchCanonicality batchcanon.Reader

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] This gate reaches coin selection and explicit forfeit selection, but handleForceUnroll never consults BatchCanonicality, and fraud routes straight into that path. A VTXO with missing/reorged lineage can still enter UnilateralExitState and create proof/sweep side effects. We need the same lineage check at the unroll point of no return, with explicit behavior for objectively invalidated lineage.

Comment thread round/transitions.go
consumerEdges = append(consumerEdges, batchcanon.ConsumerEdge{
ConsumedVTXO: outpoint,
ConsumerBatch: batchTxID,
ExpectedRevision: consumedVTXO.BusinessRevision + 1,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] We register this future business revision before ForfeitConfirmedToVTXO has persisted the matching ForfeitedBy(consumer) marker. If the consumer becomes ConflictFinalized first, ResolveConsumerEdge defers; the later MarkForfeited doesn't redrive the edge. That can strand the VTXO until restart. We should redrive after MarkForfeited succeeds and add the ordering test where terminal conflict arrives before the outbox marker.

i)
}

prevOut := input.GetPrevOut()

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] We're authenticating the outpoint/order against TxIn, but not the supplied prevout value/script. Any nonempty script and nonnegative value passes, and that script later keys the spend notifier. Unless the indexer is explicitly trusted for these fields, we need to resolve/prove the prevout against chain truth and test a changed nonempty script/value. At minimum, the PR should state this trust boundary instead of calling the payload authenticated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants