Repository navigation
Conversation
Core records (worker jobs, notifications, export envelopes, measured operations) carried loose booleans and status strings that every consumer re-interpreted, so a settled job could be re-queued and a UI badge could disagree with an API payload. - core/lifecycle: declarative state machine (states, terminal states, (from, event) -> to rules) with stable rejection codes (unknown_record_kind, unknown_state, unknown_event, invalid_transition, terminal_state) and deterministic transition records - core/lifecycle/records.ts: the four core record kinds plus stateView(), the single source for UI labels/tones and API state responses - core/workers: every status change now goes through the worker_job table; retryJob/deadLetter return a Result and refuse a settled job; JOB_STATUSES is derived from the table - core/notifications: state is stored and the read flag is derived from it, legacy entries migrate on read, and a second markRead is refused - core/contract: locks the lifecycle.state view - docs/LIFECYCLE.md, WORKERS.md, API_CONTRACT.md, ARCHITECTURE.md - tests: every allowed transition for every record kind, eight rejected moves, telemetry attribution and determinism
Retried submissions could create duplicate jobs, duplicate export artifacts
and duplicate user-facing effects. Opt-in idempotency keys now make those
writes safe to retry.
- core/idempotency: key validation, canonical request digests, persisted
outcomes (in-flight/completed/failed) behind a pluggable storage adapter,
replay counting, TTL sweep, and withIdempotency for single call sites
- strict refusals: idempotency_key_missing / _invalid / _expired / _conflict,
idempotency_in_flight, idempotency_not_found
- a replayed request never re-runs the work: a completed key returns the
recorded response, a failed key returns the recorded failure
- core/workers: submit() is the guarded enqueue and returns
{ job, replayed, record }; enqueue() is unchanged
- core/export: exportRecords accepts idempotencyKey and replays the envelope
- recorded responses pass through telemetry redaction; the envelope's own
key stays verbatim so the record is readable on the next attempt
- core/contract locks idempotency.record
- docs/IDEMPOTENCY.md, WORKERS.md, EXPORTS.md, API_CONTRACT.md
- tests: replay after success, replay after failure, thrown error, key
collision, expired key, in-flight claim, sweep, persistence and redaction
Stored records and the numbers shown to users could silently drift apart: a cached balance disagreed with the ledger, a job sat in retrying for an hour, an envelope outlived its retention, an idempotency claim never settled. - core/reconciliation: 13 declared invariants across the four drift kinds (missing, duplicate, stale, inconsistent), each with repair guidance that is reported and never applied - checks cover balances vs ledger references, worker job state/budget/dedupe/ progress, idempotency claim freshness, envelope count and retention, notification state and identity, unsettled operations - amounts compared as decimal strings with BigInt, never floats - report is pure: injected clock and snapshot providers, no write path, findings scrubbed of embedded secrets, output passed through redact() - core/reconciliation/job.ts registers reconciliation.dry_run on the worker framework plus a synchronous runNow(), with a reconciliation_run lifecycle - scripts/verify-reconciliation.mjs (npm run verify:reconciliation, wired into npm run check) fails if an invariant is declared but unchecked, if a kind disappears, or if the engine grows a write path - core/contract locks reconciliation.report - docs/RECONCILIATION.md, WORKERS.md, API_CONTRACT.md, ARCHITECTURE.md - tests: one scenario per drift kind, deep-frozen snapshot proving no mutation, unreadable provider, and read-only CLI guard
When a job dead-letters or an export goes out repository-wide, the question is always the same: who did this, on what, and why? Telemetry answers it for dashboards, but telemetry is best-effort, sampled and dropped under load. - core/audit: one structured event per sensitive action, with actor, scope, target, reason, outcome (allowed/denied), before/after, at and correlationId - SENSITIVE_ACTIONS declares the seven covered actions; every one is emitted by a domain boundary, never by a UI button - before/after are primitives only, capped at 12 keys and 160 chars, passed through redact() and an embedded Stellar-secret scrub, so an event cannot carry a payload - append-only: no update, no per-event delete, retention prune of the oldest entries; clock, id factory, storage and limit are injected - maintainer query and export (ndjson/json/csv) with filtering; a user actor is confined to its own own-scope events, anything wider is audit_denied - refusals are recorded too: denied transitions, denied maintainer exports and released idempotency claims all carry their error code - which lifecycle moves audit is explicit: terminal moves always, retry/fail only when a human asked; retryJob and deadLetter pass audited: true - scripts/verify-audit.mjs (npm run verify:audit, wired into npm run check) fails on an undeclared or unemitted action, a missing update/delete, or a removed redaction - core/contract locks audit.event - docs/AUDIT.md, API_CONTRACT.md, ARCHITECTURE.md, README - tests: attribution, secret and payload refusal, key and value caps, append-only, retention, storage adapter, maintainer/user authorization, csv/json/ndjson export, and one scenario per emitting boundary
|
@Dannyswiss1 Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits. You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀 |
|
The PR is mergeable and has no reported failing checks, but the available account does not have repository merge permission. Please have a repository maintainer review and merge it. |
Wilfred007
pushed a commit
to bytebenders1/utix
that referenced
this pull request
Oct 21, 2026
…cycle-idempotency-reconciliation-audit
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Utix had four related gaps in
core/, all of them visible whenever somethingwent wrong: record state was implicit in ad-hoc booleans, a retried write
repeated its side effect, stored numbers could silently disagree with the
ledger, and there was no way to answer "who did this, and why?".
This pull request implements issues #43–#46 as four
core/modules, each withtests, a contributor doc and a CI guard:
core/lifecycle— one deterministic state machine(
applyTransition) drives every core record:worker_job,notification,export_envelopeandoperation. States, allowed events and terminal statesare data, not code. The worker queue and the notification store now move
state only through the table, so an illegal move is a typed
Result(
unknown_state,unknown_event,invalid_transition,terminal_state)instead of a mutated object.
stateView()gives the UI one label/tone source.core/idempotency— a persisted claim → result → replay record witha canonical request digest.
WorkerFramework.submit()is idempotent(including the unknown-key replay path), and
exportRecordsaccepts anidempotencyKey. Conflicts returnidempotency_key_conflict; an in-flightclaim returns
idempotency_in_flight; an expired key frees the key insteadof replaying a stale result.
core/reconciliation— a read-only dry run that reports driftinstead of repairing it. 13 declared invariants across the four drift kinds
(
missing,duplicate,stale,inconsistent) over balances, worker jobs,idempotency claims, export envelopes, notifications and operations, each
finding carrying repair guidance for a human. Amounts are compared as
decimal strings with
BigInt, never floats. Registered as thereconciliation.dry_runworker operation.core/audit— an append-only trail for the seven sensitive actions,written at domain boundaries rather than by the UI. Every event carries
actor, scope, target, reason, outcome (
allowed/denied) and correlation id.before/afterare primitives only, capped, and passed throughredact()plus an embedded Stellar-secret scrub, so an event cannot carry a payload.
Maintainers get authorized
queryandexport(ndjson/json/csv); a useractor is confined to its own
own-scope events.Closes #43
Closes #44
Closes #45
Closes #46