Oracle consensus governance: payout holds, point-in-time reconstruction, mirror detection, provider offboarding by Sundriveauto · Pull Request #474 · Refract-Protocol/refract-backend

Sundriveauto · 2026-10-21T15:45:39Z

Summary

Adds a consensus evidence layer for oracle feeds and builds four governance features on it. Each issue is a separate commit:

Commit	Issue	What it adds
`0531b90`	#272	Automatic payout holds when a triggering decision's evidence is incomplete
`6c3b0c3`	#271	Point-in-time consensus reconstruction (API + CLI)
`da81ed5`	#273	Detection of cloned or mirrored sources via reading fingerprints, with independence penalties
`c8212af`	#274	Provider offboarding: retire, revoke credentials, purge payloads, keep evidence

Everything lives in src/oracle/consensus/. Operator docs are in docs/runbooks/oracle-consensus.md.

Shared foundation (first commit)

Append-only evidence ledger (evidence-ledger.ts). Every fact is an immutable event: source versions, quorum-policy versions, readings, signature results, decisions, holds, penalties, purges and audit entries.
- Events persist as JSONL at EVIDENCE_LEDGER_PATH and are reloaded on boot. The CLI reads the same file.
- There is no update or delete. Corrections are new events.
- Timestamps are strictly increasing, so "everything recorded at or before T" is an exact prefix of the ledger.
Content-addressed payload store. Raw bodies are stored by the sha256 of their canonical JSON. The hash is validated before it is used as a file name.
Pure, version-aware lookups (ledger-view.ts) and a pure consensus engine (consensus.engine.ts: window filter, eligibility, quorum, weighted median). Live decisions and historical reconstruction use the same functions, so there is one definition of "state at T".
RBAC (consensus-role.guard.ts). OPS_API_TOKEN grants the operator role (read/write). Tokens in AUDITOR_API_TOKENS (CSV) grant the read-only auditor role. Comparison is timing-safe.

#272 Payout holds on incomplete or immature evidence

Precheck. assessEvidenceCompleteness() is a pure function that returns structured gaps:
- missing_raw_payload
- insufficient_sources: fewer independent contributors than the policy's expectedSources
- source_in_warmup
- signature_pending
Gate. ClaimService.processPayout now calls ClaimHoldService.gate() before settling. A triggered claim with gaps moves to payout_held with the gap list and a human-readable reason. It is not paid and not silently denied.
Explicit state machine. payout_held → released → settled, and payout_held → escalated → approved | denied, with extend_hold returning a claim to payout_held. A hold can't be mistaken for a denial.
Re-evaluation runs every CLAIM_HOLD_REEVALUATE_INTERVAL_MS:
- It releases the hold once a decision has no gaps and is still triggered.
- It escalates on any of: deadline_exceeded (CLAIM_HOLD_MAX_DURATION_MS), verdict_contradicted (newer evidence says "don't pay"; never auto-denied), or policy_expired_while_held.
Operator queue.
- GET /api/v1/ops/consensus/holds?status=escalated and GET /holds/:holdId
- POST /holds/:holdId/disposition with a required actor and note
- npm run oracle:ops -- holds
Ledger and audit. Every transition is a ledger event plus an audit event. The queue is rebuilt from the ledger after a restart.
Edge cases:
- Re-scans are idempotent: one hold per policy and decision.
- If the ledger can't be written, the gate fails closed: the payout is blocked and retried on the next scan.
- One failing hold can't stall the others or the scan cycle.
No behaviour change for current feeds. Feeds without a quorum policy are not under consensus governance and settle exactly as before.

#271 Point-in-time consensus state reconstruction

reconstructConsensusState(events, { feedId, at | ledgerSeq }) is a pure function of persisted events. It never uses live config, caches or provider calls. It returns:
- the quorum policy in force at T
- per-source version, status, eligibility and reason
- the contributing readings, with value, payload hash and signature status
- the recomputed decision, or no_decision with a reason
- the decision actually recorded at the time
- a consistency field: matches_recorded, differs_from_recorded or no_decision_recorded_at_t
How T is resolved. Only events recorded at or before T are visible. ledger=L resolves T to the last event stamped with ledger ≤ L and excludes events stamped with a later ledger.
Stable output. The result doesn't change as the ledger grows or after payload bodies are purged.
API: GET /api/v1/ops/consensus/feeds/:feedId/state?at=<ISO>|ledger=<seq> (operator or auditor).
CLI: npm run oracle:ops -- reconstruct --feed … --at …. It shares the canonical serialiser with the API, and a test asserts the two outputs are byte-identical.
Edge cases covered: a time between readings, superseded decisions, sources added after T, policy changes after T, no policy or no sources at T, a ledger before any evidence, and confirmation that reads never mutate state.
Property test. 25 seeded random histories, each with source, policy and status churn and failed signatures. Every recorded live decision is reconstructed and must come out matches_recorded. A mutation check confirmed the suite fails if reconstruction can see events recorded after T. Decisions recorded before the independent flag existed still reconstruct as matches_recorded.

Sample reconstruction output (CLI)

{"consistency":"matches_recorded",
 "decision":{"outcome":"decision","policyVersion":1,"reason":"triggered: 0.915 lt 0.95","triggered":true,"value":0.915,
   "contributing":[{"effectiveWeight":1,"independent":true,"readingId":"rd_385163c19445f885b2929f4d","sourceId":"binance","value":0.918}, "…"]},
 "contributingReadings":[{"ledgerSeq":51234500,"payloadHash":"9bc27804…","readingId":"rd_c1af435da5ad35e4d7c655bc","signature":"pending","sourceId":"kraken","value":0.915}, "…"],
 "independencePenalties":[],
 "knowledgeCutoffSeq":11,
 "query":{"at":"2026-10-21T14:00:03.002Z","feedId":"StablecoinDepeg","ledgerSeq":null},
 "quorumPolicy":{"expectedSources":3,"maxReadingAgeMs":300000,"minSources":2,"minTotalWeight":2,"trigger":{"op":"lt","threshold":0.95},"version":1, "…"},
 "recordedDecision":{"decisionId":"dc_3030dc24f264636fa17b7ae9","atIso":"2026-10-21T14:00:03.002Z","outcome":"decision","triggered":true,"value":0.915, "…"},
 "resolvedAtIso":"2026-10-21T14:00:03.002Z",
 "sources":[{"eligible":true,"immature":false,"independent":true,"mirrorOf":null,"providerId":"binance","reason":null,"sourceId":"binance","status":"active","version":1,"weight":1, "…"}, "…"]}

Pretty-printed and trimmed (…). The real output is a single canonical line.

#273 Cloned or mirrored source detection

Fingerprint. Each ingested reading stores sha256(feedId, canonical value, time bucket). Values are canonicalised to 12 significant digits, so formatting and float noise don't defeat matching.
Batch detection (MIRROR_SCAN_INTERVAL_MS), off the hot path. It compares every source pair within a feed over a sliding window:
- Each source contributes its last value per bucket.
- Pairs are compared at the best lag up to MIRROR_MAX_LAG_BUCKETS, so delaying a copy doesn't hide it.
- Values match within MIRROR_RELATIVE_TOLERANCE, so small jitter doesn't hide it either.
- Only informative buckets count, meaning buckets where a series moved. A flat market proves nothing.
- A pair is flagged only on sustained overlap: enough informative samples, a high enough match ratio, and enough distinct values (which guards categorical and low-cardinality feeds).
Penalty. A flagged pair gets a persisted independence_penalty with the window, lag, statistics and up to 10 matched reading pairs as evidence. Consensus then:
- multiplies the lighter source's weight by MIRROR_PENALTY_FACTOR;
- stops counting it toward the minSources quorum and toward a payout's expectedSources.
Reconstruction replays penalties as of T. A penalty clears when the pair diverges or a source is retired; quiet periods never clear it.
Operator view: GET /api/v1/ops/consensus/trust-status shows sources, declared upstream, and suspected pairs with evidence.
Metrics: refract_oracle_suspected_mirror_pairs{feed} and refract_oracle_mirror_penalty_changes_total{feed,action}.
Backtest: npm run oracle:ops -- mirror-backtest replays detection over the whole ledger and writes nothing.
Tests: an exact mirror; a near-mirror that is delayed and jittered; agreement on a flat feed; independent noisy sources tracking one market; a categorical feed that toggles; a cross-feed false-positive check; clearing; the quorum effect; reconstruction replay; the payout gate; metrics.

Backtest analysis output (synthetic: 24 h, 4 sources, "reseller" copies CoinGecko with a 15 s delay)

Pair	Max match ratio	Informative samples	Would penalise
binance / coingecko	0.2571	357	no
binance / kraken	0.2578	357	no
binance / reseller	0.2571	357	no
coingecko / kraken	0.2535	358	no
coingecko / reseller	1.0	343	yes, first flagged after one full window
kraken / reseller	0.2535	358	no

Honest pairs top out around 26% matches against the 90% threshold, so the margin is wide.

#274 Provider offboarding

POST /api/v1/ops/consensus/sources/:id/offboard runs four stages:

Retire. A new source version marks the source retired. It leaves consensus, and new readings from it are rejected with 409. It is never deleted.
Revoke. The stored credential is deleted through a pluggable SourceCredentialStore: AWS Secrets Manager (SOURCE_CREDENTIALS_BACKEND=aws-secrets-manager) or in-memory.
Purge. Raw payload bodies past the licence's retentionDays are deleted. Hashes, values, readings and decision references all stay.
Record. Ledger and audit entries capture the actor, the reason and the purge scope.

Details:

Dry run. dryRun: true or npm run oracle:ops -- offboard-plan gives the exact plan. Both use the same pure planOffboarding(), and a test asserts that execution follows the dry-run plan exactly.
Retention rules. Bodies behind an open payout hold (dispute) are retained, and so are bodies shared with an active source. Bodies still inside retention are purged later by a scheduled sweep.
Quorum guard. If retiring the source would drop its feed below quorum, the call returns 409 unless acknowledgeQuorumLoss is set.
Resumable. A partial failure is resumable and idempotent under the same offboarding id.
Re-onboarding appends a new version, so each historical instant maps to exactly one configuration.
Verification. Post-purge reconstruction of past decisions is byte-identical (tested). GET /payloads/:hash shows whether a body is held, whether it matches its hash, or when and why it was purged.

Dry-run output (CLI)

{
  "alreadyRetired": false,
  "credential": { "action": "revoke", "credentialRef": "refract/sources/acme-data" },
  "feedId": "StablecoinDepeg",
  "generatedAt": "2026-10-21T15:00:00.000Z",
  "openDisputes": [],
  "payloads": {
    "alreadyPurged": 0,
    "purge":  [{ "hash": "08aee184…f668d", "storedAt": "2026-10-01T00:00:04.000Z" }],
    "retain": [{ "hash": "4baf00b7…fbe1aa", "purgeAfter": "2026-10-25T00:00:04.000Z", "reason": "within_license_retention", "storedAt": "2026-10-18T00:00:04.000Z" }]
  },
  "preserved": { "decisions": 2, "payloadHashes": 2, "readings": 2 },
  "providerId": "acme-data",
  "quorum": { "dropsBelowQuorum": false, "minSources": 2, "policyVersion": 1, "remainingEligibleSources": 3 },
  "retireAsVersion": 2,
  "sourceId": "acme"
}

Type of change

Checklist

I read CONTRIBUTING.md
Tests added/updated for the change
Local gate passes (lint / typecheck or fmt + clippy + test / build). New code is clean, but the repo-wide build already fails on main; see below.
No secrets, keys, or .env values committed
Behavioural changes are documented (docs/runbooks/oracle-consensus.md)

Verification

New suites: 4 suites, 63 tests, all passing. Each commit also type-checks and passes its own tests on its own (22 → 35 → 51 → 63 tests).
- claim-hold.service.spec.ts
- reconstruction.spec.ts, which includes HTTP tests via @nestjs/testing + supertest for RBAC, byte-identical API vs CLI output, and the offboarding dry run
- mirror-detection.spec.ts
- offboarding.spec.ts
Full npx jest, compared per suite with --json:
- Base 3943531: 23 tests pass; 5 suites pass and 33 fail.
- This branch: 86 tests pass; 9 suites pass and the same 33 fail.
- The only per-suite changes are the 4 new passing suites. No regressions.
Type-check: a strict tsc over src/oracle/consensus/**, scripts/oracle-ops.ts and src/metrics/metrics.service.ts is clean. eslint on all changed files is clean.
CLI: npm run oracle:ops -- reconstruct | holds | mirror-backtest | offboard-plan were run end to end through ts-node. The samples above come from those runs.

Notes for reviewers

Pre-existing breakage, not touched here. npm run typecheck and npm run build already fail on main: 37 syntax errors in policy/policy.service.ts, claim/claim.scheduler.ts and health/health.controller.ts. 33 test suites already fail to compile, including claim.service.spec.ts. There are also missing modules (./auth, ./claims, ./redis, ./audit), and OpsAuthGuard reads an ops config key that configuration.ts doesn't define. That is why the consensus endpoints have their own small role guard instead of OpsAuthGuard. CI will stay red until those are fixed.
ClaimService integration. The gate is injected with @Optional(), so existing wiring keeps working. I verified hold → release → settle-once through ClaimService.processTriggered() with the broken neighbours mocked. That spec isn't committed, because it can't type-check until policy.service.ts and claim-settlement.service.ts are fixed. I can add it once they are.
Includes commit 3943531 ("symmetrical"). It was already on this branch and is not in upstream main, so it appears in this PR's diff.

Not done in this PR:

Postgres-backed ledger. The ledger is persisted as an append-only JSONL file plus a payload directory. No schema.sql or migration was added, because the repo's Postgres wiring isn't functional on main (for example, PG_POOL is imported but never exported). The ledger interface is small (append, events), so a Postgres adapter can replace it.
Feeding live readings. OracleService still produces single-source readings. Multi-source readings enter through POST /api/v1/ops/consensus/readings. Until a feed has a quorum policy, it isn't governed and behaviour is unchanged.
Real historical data. The Detect Cloned or Mirrored Sources via Payload Fingerprinting #273 backtest and the Implement Graceful Provider Offboarding with Data Purge and Evidence Continuity #274 staging check ran on synthetic ledgers. The repo has no multi-source history yet. Re-run mirror-backtest on real data before enabling penalties in production.
Provider portals. Revoking keys inside third-party portals is out of scope per Implement Graceful Provider Offboarding with Data Purge and Evidence Continuity #274. It is a manual step in the runbook.

Closes #271
Closes #272
Closes #273
Closes #274

🤖 Generated with Claude Code

…t-Protocol#272) Adds the consensus evidence layer (append-only ledger, versioned source and quorum-policy config, pure consensus engine). On top of it, adds an evidence-completeness precheck that runs before settlement. A triggered claim whose decision has gaps (missing raw payload, fewer contributors than expected, a warm-up source, a pending signature) moves to `payout_held` with a structured gap list. It is not paid and not silently denied. Held claims are re-evaluated on a configurable cadence. They release automatically once a decision is complete and still triggered. They escalate to an operator queue on deadline, on a contradicting verdict, or when the policy expires while held. All transitions are ledger and audit events, so the queue survives restarts. The gate fails closed: if the ledger cannot be written, the payout is blocked and retried on the next scan. Feeds without a quorum policy are not governed and settle exactly as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…rotocol#271) Adds reconstructConsensusState(), a pure function that rebuilds a feed's consensus state at an ISO timestamp or a ledger sequence from persisted evidence only. The result includes the quorum policy version in force, per-source eligibility, the contributing readings, the recomputed decision, and the decision actually recorded at the time. Only events recorded at or before T are visible. Ledger timestamps are strictly increasing, so the answer is stable as the ledger grows and after payload bodies are purged. The API (GET /api/v1/ops/consensus/feeds/:feedId/state, operator or auditor role) and the CLI (`npm run oracle:ops -- reconstruct`) share the function and a canonical serialiser, so their output is byte-identical. A seeded property test checks that reconstruction reproduces every live decision across randomised histories. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ct-Protocol#273) Each ingested reading now stores a fingerprint: sha256 of the feed, the canonical value and a coarse time bucket. A periodic batch scan, run off the hot path, compares every source pair on a feed over a sliding window. It aligns the pair at the best lag, so a delay can't hide a copy, and counts only buckets where the series moved, so flat markets prove nothing. A pair is flagged only on sustained near-exact agreement across several distinct values. A flagged pair gets a persisted independence_penalty with the window, lag, statistics and matched-reading evidence. Consensus then scales down the lighter source's weight. That source also stops counting toward the minSources quorum and toward a payout's expectedSources. Reconstruction replays penalties as of T. Penalties clear when the pair diverges or a source is retired. Results are exposed in a trust-status view, as Prometheus metrics, and in a read-only `oracle-ops mirror-backtest` command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ontinuity (Refract-Protocol#274) Retiring a source is now a designed operation and never a delete: (a) a new source version marks it retired, removing it from consensus and rejecting new readings; (b) its stored credential is deleted through a pluggable credential store (in-memory, or AWS Secrets Manager); (c) raw payload bodies older than the licence retention are purged while every hash, value and decision reference stays in the ledger; (d) each step is recorded with actor, reason and purge scope. planOffboarding() is a pure dry run shared by the API and `oracle-ops offboard-plan`. Execution follows the same plan. Bodies behind open payout holds, or shared with an active source, are retained. Bodies still inside retention are purged later by a scheduled sweep. Removing a source that would drop a feed below quorum needs explicit acknowledgement. Partial failures are resumable and idempotent under the same offboarding id. Re-onboarding appends a new version, so each historical instant maps to one configuration. Past decisions reconstruct byte-identically after a purge. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

drips-wave · 2026-10-21T16:02:25Z

@Sundriveauto Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

…nce-consensus-holds # Conflicts: # package.json # src/app.module.ts # src/claim/claim.service.ts # src/oracle/oracle.controller.ts # src/oracle/oracle.scheduler.spec.ts # src/oracle/oracle.service.spec.ts # src/oracle/oracle.service.ts

Sundriveauto and others added 5 commits October 21, 2026 14:53

symmetrical

3943531

OxDev-max merged commit e93b448 into Refract-Protocol:main Oct 23, 2026
0 of 3 checks passed

Navigation Menu

Repository navigation

Oracle consensus governance: payout holds, point-in-time reconstruction, mirror detection, provider offboarding - #474

Oracle consensus governance: payout holds, point-in-time reconstruction, mirror detection, provider offboarding#474
OxDev-max merged 6 commits into
Refract-Protocol:mainfrom
Sundriveauto:feat/oracle-governance-consensus-holds

Sundriveauto commented Oct 21, 2026

Uh oh!

drips-wave Bot commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Sundriveauto commented Oct 21, 2026

Summary

Shared foundation (first commit)

#272 Payout holds on incomplete or immature evidence

#271 Point-in-time consensus state reconstruction

#273 Cloned or mirrored source detection

#274 Provider offboarding

Type of change

Checklist

Verification

Notes for reviewers

Uh oh!

drips-wave Bot commented Oct 21, 2026

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants