Skip to content

refactor: add support for postgres store(s) - #28

Merged
frrist merged 3 commits into
mainfrom
frrist/stores/postgres
Apr 28, 2026
Merged

refactor: add support for postgres store(s)#28
frrist merged 3 commits into
mainfrom
frrist/stores/postgres

Conversation

@frrist

@frrist frrist commented Apr 18, 2026

Copy link
Copy Markdown
Member

Closes #27.

Adds a PostgreSQL backend alongside the existing DynamoDB and in-memory backends so sprue can run self-hosted as part of the Forge stack.

Summary

Thirteen new postgres/ store implementations (one per domain interface under pkg/store/), a goose-based migrations runner, an fx module that wires them in, a storage.type backend selector in config, and a docker-compose dev stack built around Postgres + MinIO. The default backend is now postgres.

The UCAN handlers, routing, service layer, and external clients are untouched — the Store interfaces are the seam and no interface changed in this PR.

Scope — in

Store implementations

New pkg/store/<domain>/postgres/store.go for each of the 13 domains, matching the existing aws/ and memory/ layout:

agent, blob_registry, consumer, customer, delegation, metrics (both SpaceStore and admin Store), replica, revocation, space_diff, storage_provider, subscription, upload.

The three stores that also persist blob payloads (agent, delegation, upload) keep their S3 side unchanged — only the DynamoDB metadata half moved to Postgres.

Schema + migrations

  • internal/migrations/sql/ — one goose migration per store (00001_init.sql00013_agent_index.sql).
  • internal/migrations/migrations.go — embedded FS + a goose.UpContext runner, with a zap adapter for goose logging.
  • Migrations run automatically at startup. storage.postgres.skip_migrations: true disables them.

fx wiring

  • internal/fx/store/postgres/provider.go — new postgres.Module that provides the pool, the migrated pool, the S3 client (reused from the AWS module for the three stores that need it), and every Postgres-backed store.
  • internal/fx/app.goAppModule now switches on cfg.Storage.Type (memory | postgres | aws) and wires the matching store module. An empty Type is treated as postgres so tests constructing Config literals don't have to set it.

Config

  • internal/config/config.go — replaces the old InMemoryStores boolean with a StorageConfig.Type selector (memory | postgres | aws, default postgres). Adds PostgresConfig (DSN, max/min conns, skip_migrations). DynamoDB and S3 settings moved under StorageConfig for symmetry.
  • Env var bindings follow the existing SPRUE_* convention: SPRUE_STORAGE_TYPE, SPRUE_STORAGE_POSTGRES_DSN, SPRUE_STORAGE_POSTGRES_MAX_CONNS, SPRUE_STORAGE_POSTGRES_SKIP_MIGRATIONS, etc.

Tests

  • internal/testutil/postgres.go — spins up a throwaway Postgres container via testcontainers-go, applies migrations, and returns a *pgxpool.Pool.
  • Every existing *_test.go store test matrix gains a postgres case alongside memory / aws — same assertions run against all three backends.
  • internal/fx/app_test.go TestWireApp exercises Postgres wiring end-to-end.

Dev stack

  • docker-compose.yaml (top level) — Postgres + MinIO + sprue service, healthchecks, sprue env vars pre-wired to the Postgres backend.
  • Dockerfile, Makefile, config.example.yaml, README.md, CLAUDE.md updated for the new backend + config surface.

Design decisions

pgx + pgxpool, no ORM, no sqlc (yet)

Queries are hand-written SQL via pgx.Pool.QueryRow / Exec. Reasons:

  • Stores do cross-domain work (e.g. blob_registry coordinating with consumer, space_diff, metrics) — explicit coordination is already the pattern, an opaque ORM would fight it.
  • Sqlc was considered but deferred: getting the first backend working with typed queries is lower-risk with direct pgx, and sqlc can be layered on later without changing the store interfaces.

goose for migrations, embedded from the binary

Matches the convention already set by Guppy in the Forge stack. Migrations are embedded via //go:embed sql/*.sql so there's nothing to copy at deploy time. Running at startup (and skippable via config) keeps operator workflow simple for the early self-hosted deployments this unblocks.

Single shared schema, one migration per store

One Postgres schema holds all 13 stores' tables. Per-store schemas would buy nothing (no access-control story here) and would force every cross-domain query to be schema-qualified. One migration per store keeps ownership boundaries clear within the shared schema.

MigratedPool as a lifecycle ordering primitive

MigratedPool is a typed wrapper around *pgxpool.Pool whose sole job is to order fx lifecycle hooks. NewMigratedPool registers the migration OnStart hook; every store constructor depends on *MigratedPool (not *pgxpool.Pool), so the fx dependency graph resolves migrations first and — since OnStart hooks fire in registration order — guarantees every store's Initialize hook sees a fully migrated schema. No explicit sequencing code, no ad-hoc "wait for migrations" checks.

Postgres is the default backend

storage.type defaults to postgres (with a sensible default DSN for the docker-compose stack). Empty Type is also treated as postgres in AppModule so test-constructed configs don't need to set it. AWS stays available via storage.type: aws; memory stays available for tests and local experimentation.

Agent store: removed the async job queue (commit 2e66d37)

The Postgres agent store originally mirrored the AWS store by using go-libstoracha/jobqueue to fan out writes (one S3 put + N index inserts per Write). That design has durability holes independent of the Postgres backend — the job handler runs on a background context, so cancellation and shutdown can leak or drop in-flight work, and the separate jobs offer no atomicity across the fan-out. Full analysis and the AWS counterpart are tracked in #33.

For the Postgres store specifically, the job queue has been replaced with:

  1. Caller-context synchronous writes. S3 put and Postgres exec both run on the caller's ctx; cancellation propagates.
  2. Payload first, then index. S3 is written before the index row so a partial failure leaves (at worst) an unreferenced S3 object — dangling index pointers to missing payloads are no longer reachable.
  3. Single batched INSERT ... VALUES (…), (…) ON CONFLICT (task, kind) DO UPDATE. All index rows commit atomically in one statement, preserving the AWS overwrite semantic for cross-call conflicts.
  4. Intra-message dedup. agent.Index can yield the same (task, kind) pair more than once (e.g. a receipt re-yielding its invocation via Ran()); rows are deduped in Go before the INSERT. For kind="in" this is provably safe (task CID is the invocation CID). Potential ambiguity for kind="out" across different receipts in one message is a known edge case — see discussion in Remove jobqueue from agent.Store — unneeded complexity with durability risks #33; not changed here because it exists identically in the AWS store and resolving it is orthogonal to the Postgres port.

Scope — out

  • S3 usage is unchanged. Agent messages, delegations, and upload shards still live in S3/MinIO.
  • UCAN handlers, routing, piriclient, indexerclient, service-layer logic — none changed. Handlers depend on store interfaces, not implementations.
  • AWS agent-store jobqueue removal. The same durability issues exist in pkg/store/agent/aws/store.go; tracked in Remove jobqueue from agent.Store — unneeded complexity with durability risks #33. Not touched here so this PR stays scoped to the Postgres port.

Gates before merge (still-draft items)

  • Corresponding changes in smelt: deps: support postgres deployed sprue smelt#22
  • Better documentation (README / operator notes for running the Postgres backend; this description is a first pass)
  • Further schema-design evaluation. Migration infrastructure is in place because it'll be needed eventually, but given there's no production deployment yet, consolidating the per-store migrations into a single pre-production baseline should be considered so the initial schema ships clean.

Open questions

  • Deprecation timeline for the AWS (DynamoDB) backend — keep indefinitely, mark deprecated, or remove once Postgres is in production?
  • Schema name given the Forge rename (sprue, forge, something else) — currently uses the default public schema.

@frrist
frrist marked this pull request as ready for review April 23, 2026 18:37
@frrist
frrist merged commit 4990aff into main Apr 28, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace DynamoDB with PostgreSQL

2 participants