Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 58 additions & 94 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,120 +4,84 @@ All notable changes to this project are documented in this file. The format is b

Pre-1.0 note: while `pg_durable` is in major version `0`, minor releases may include breaking changes.

## [0.2.9] - Unreleased

The changes below landed after the v0.2.8 tag and are not part of that release.

Runtime shutdown and server-side quiescence remain separate known limitations:
see [deferred guarantees](docs/multi-database-installation.md#release-blockers).
## [0.2.9] - 2026-10-07

> **Upgrade:** Install/restart the new binary and wait for the control runtime
> to become ready before creating satellites. Existing control schemas from
> 0.2.2 onward remain supported; run `ALTER EXTENSION pg_durable UPDATE TO '0.2.9'`
> to add the new SQL APIs. Reapply `df.grant_usage(role, include_http => true)`
> to enable typed HTTP endpoints for existing HTTP users.
> Installations previously built without HTTP support must set
> `pg_durable.http_security = 'disabled'` before restarting to keep HTTP blocked.
> See [upgrade guidance](docs/upgrade-testing.md#v028--v029).

> **Known limitations:** DROP is not a cancellation or quiescence acknowledgment.
> Already-admitted remote SQL and HTTP requests may finish after source removal;
> the final admission check and send are not atomic with DROP. Runtime task
> shutdown and confirmed PostgreSQL backend cancellation remain deferred.
> See [deferred guarantees](docs/multi-database-installation.md#release-blockers).

### Added

- **Multi-database installations:** install the shared control runtime first,
then satellites with local `df` metadata, grants, RLS, endpoint catalogs and
secret mappings. SQL defaults to its origin unless explicitly targeted.
Satellite engine IDs include database OID and installation UUID; legacy
control IDs and recorded payloads remain unchanged.
- **Bounded origin connections:** `pg_durable.max_origin_connections` defaults
to `12`. Active routes reserve one metadata slot; idle satellites retain no pool.

- **HTTP body policies (#376):** opt-in request and response byte caps,
- **Multi-database installations (#401):** satellites share one control runtime
while keeping local `df` APIs, metadata, grants, RLS, endpoints and credentials.
SQL defaults to the submitting database unless explicitly targeted and retains
autocommit. `pg_durable.max_origin_connections` bounds origin connections
(default `12`); idle satellites retain no connection pool.
- **HTTP body policies (#397):** opt-in request and response byte caps,
`inline`/`metadata`/`discard`/`sink` response modes, and response-header selection
through `df.with_http_options`, for ordinary and multipart requests.
Response limits are enforced while reading, including after decompression.
Metadata, discard, and sink modes keep response bodies out of durable results
and 5xx error previews. Table sinks store raw bytes under the submitting role's
permissions and return a committed row reference with a fresh key per attempt.
Existing defaults and HTTP signatures are unchanged.

- **`pg_durable.http_allowed_domains` (#375):** a restart-only GUC that replaces the HTTP and multipart domain allow-list with exact hostnames and `*.domain` patterns. Defaults to Azure service subdomains and `api.github.com`; an explicit empty list denies all domains in restricted mode. It cannot enable disabled HTTP or relax the other safeguards.

- **Explicit secret bindings:** `df.secret(server, key)` returns a JSONB
descriptor for named header/query/form fields through `df.with_http_options`.
Individual `"secret.<key>"` user-mapping options support per-key addition,
rotation and removal. Literal form data
stays separate from references; activities encode fields and resolve credentials
under the submitting role without scanning payloads for markers. Named-secret-only
servers may omit `base_url` with `auth_scheme 'none'`.
- **Endpoint HTTP requests:** `df.endpoint(server, path)` returns a typed
`df.http_endpoint` value accepted by `df.http` and `df.http_multipart`.
TEXT destinations remain URLs, never serialized endpoint references.
Activities resolve per-role credentials,
enforce server `USAGE`, preserve the configured base URL and reject routing or
credential overrides. Existing HTTP signatures and raw-URL workflow inputs
remain unchanged; the grant/revoke helpers cover URLs and endpoints together.
General body secret interpolation is deferred.
- **Endpoint credential catalog:** handler-less `pg_durable_fdw`, a closed-set
option validator, and per-user catalog resolution for unauthenticated, bearer,
named-header and query-string endpoint authentication. FDW creation authority
is delegated with native grants. User mappings remain plaintext and may be
included in dumps; `DROP EXTENSION ... CASCADE` removes dependent endpoints
and mappings. Catalogs live in the origin database independently of SQL targets;
each request uses a consistent caller-authenticated snapshot and shares the SQL
connection budget, releasing its connection before HTTP I/O.
- **HTTP options helper (#380):** adds `df.with_http_options(fut, options)` for
single HTTP and multipart nodes. `secret_bindings` and `form_fields` configure
references and literal form data. SQL `NULL` and an empty JSON object return
the original node unchanged; other values and unsupported keys raise an error.
Existing installations receive the helper through the 0.2.8 to 0.2.9 extension
upgrade.
permissions in the origin database or the workflow's explicit database target;
applications own sink-row retention.
- **HTTP endpoints and secrets (#383):** `df.endpoint(server, path)` and
`df.secret(server, key)` use origin-local foreign servers and per-user mappings
for endpoint authentication and explicit header/query/form secret bindings.
Existing TEXT destinations remain URLs. General body secret interpolation is
not supported. User mappings are plaintext and may appear in database dumps;
`DROP EXTENSION ... CASCADE` removes dependent servers and mappings.
- **HTTP options helper (#380):** `df.with_http_options(fut, options)` configures
a single HTTP or multipart node without changing existing HTTP signatures.
- **Configurable HTTP domains (#382):** `pg_durable.http_allowed_domains`
replaces the restricted-mode allow-list with exact hosts and `*.domain`
patterns. Defaults include Azure service subdomains and `api.github.com`;
an explicit empty list denies all domains. Changes require a restart.

### Changed

- **Satellite admission and DDL:** metadata validation/access share short
transactions, and SQL retains autocommit for all routes. Source identity is
refreshed after resource waits and before SQL/HTTP dispatch; HTTP authorization
is refreshed after credential preparation. Extension-managed DROP protection
covers short metadata operations, not arbitrary activities; normal PostgreSQL
locks still apply. DROP is not a cancellation
or quiescence acknowledgment.

- **Independent-start scope:** `transaction_mode => 'new'` launches in the
caller's database, with `max_new_transaction_starts` enforced per database.
- **Shared metrics:** satellite `df.metrics()` exposes all-engine totals.
- **Independent-start scope (#401):** `transaction_mode => 'new'` launches in the
caller's database, with `max_new_transaction_starts` now enforced per database
rather than cluster-wide.
- **Shared metrics (#401):** satellite `df.metrics()` exposes all-engine totals.
Granting `with_grant => true` includes this access and local delegation.

- **HTTP startup policy (#374):** replaces the three HTTP Cargo features with
- **HTTP startup policy (#402):** replaces the three HTTP Cargo features with
the superuser-only, restart-required `pg_durable.http_security` setting:
`disabled`, `restricted` (default), or development-only `unrestricted`.
Installations previously built without HTTP support must explicitly select
`disabled` before restarting to keep HTTP blocked. Test domains are explicitly configured
through `pg_durable.http_allowed_domains`; restricted-mode SSRF defenses and
HTTP privileges remain unchanged. Invalid security-mode values prevent server
startup instead of falling back to the default. No extension SQL migration is required.
Restricted-mode SSRF defenses and HTTP privileges remain in effect.
Invalid configuration prevents server startup rather than silently falling back.
- **HTTP connection reuse (#379):** HTTP and multipart activities share one
process-wide client and connection pool, with timeouts applied per request.
Client-construction errors are cached until the background worker restarts;
ordinary request failures do not poison the shared client.
- **PGXN documentation:** limits the PGXN documentation index to the user guide,
API reference, changelog, HTTP security guide, examples overview, and a compact
PGXN-specific README. Internal documents remain in the source distribution but
no longer appear as published documentation.
- **Dependencies (#390):** updates `uuid` from 1.26.0 to 1.26.1 and `reqwest`
from 0.13.4 to 0.13.5. The latter uses `base64` 0.23.1; pg_durable's direct
`base64` dependency remains on 0.22.1, matching its existing manifest constraint.
- **Dependencies (#390, #406):** updates `uuid` to 1.26.1, `reqwest` to
0.13.5 and the direct `base64` dependency to 0.23.1. The pinned
`duroxide` 0.1.30 / `duroxide-pg` 0.1.34 pair is unchanged.

### Fixed
### Security

- **SQL name-resolution hardening (#418):** extension-owned runtime SQL
explicitly qualifies catalog objects, functions, types and operators to resist
shadowing through caller or database `search_path` settings. User-authored SQL
and its execution search path are unchanged; no schema migration is required.

### Documentation

- **Satellite graph admission and maintenance:** transient routing contention
now uses the graph retry protocol. Retention continues bounded pages promptly
and rotates between origins; retained rows do not consume its candidate budget.
Legacy orphan IDs are filtered and paged in SQL. Removed registrations are
collected once all corresponding engine work is gone, without treating
connection failures as proof of removal.

- **Satellite metadata trust:** worker-role satellite connections use catalog-first
startup name resolution. Catalog-only type, ownership and extension-membership
checks run before and after metadata locks, before any identity data is read.
Replacement views and changed object identities fail closed without evaluation;
user SQL and response-sink search paths remain unchanged.

- **Release automation (#389):** release-triggered Docker publication retries
package downloads while the package workflow attaches release assets. Manual
dispatch remains fail-fast when assets are missing.
- **E2E test isolation (#388):** extension lifecycle tests restore the shared
test user's HTTP and multipart privileges so later tests do not depend on
whether setup has run again. This is a test-only change.
- **PGXN documentation (#395):** publishes a compact landing page and indexes
user-facing guides instead of internal development documents.
- **Upgrade compatibility (#407, #413):** documents the schema guarantees,
real previous-binary replay coverage, and remaining coverage gaps.

## [0.2.8] - 2026-09-11

Expand Down
2 changes: 1 addition & 1 deletion docs/multi-database-installation.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Multi-Database Extension Installation

**Status:** Unreleased 0.2.9 integration with admission-based source fencing.
**Status:** Available in 0.2.9 with admission-based source fencing.
See [Deferred Lifetime Guarantees](#release-blockers) before destructive removal.
**Date:** 2026-09-10

Expand Down
28 changes: 24 additions & 4 deletions docs/upgrade-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -355,7 +355,7 @@ what the upgrade script handles, and any backward compatibility considerations.

The post-tag changes in #379, #388, #389, #390, and #380 belong to the 0.2.9
development cycle, not the published v0.2.8 release. Multi-database installation
identity and HTTP endpoint/secret DDL also belong in this unreleased migration;
identity and HTTP endpoint/secret DDL also belong in this migration;
the shipped 0.2.7 to 0.2.8 script must remain byte-identical.

#### Multi-database installation
Expand Down Expand Up @@ -446,7 +446,7 @@ for the separate runtime shutdown and server-side cancellation limitations.
gain binding/form fields and trusted target-database metadata. Existing HTTP
signatures, grants and legacy raw-URL activity inputs remain unchanged.
- Named credential lookup uses native catalogs, checks server `USAGE` and reads
the authenticated caller's mapping in the control database, independently of
the authenticated caller's mapping in the origin database, independently of
the SQL target. Endpoint and named-binding reads share one read-only consistent
snapshot and the existing user-connection budget, released before HTTP I/O.
No additional DDL, grant changes or replay-visible activity inputs are needed
Expand All @@ -470,7 +470,8 @@ for the separate runtime shutdown and server-side cancellation limitations.

- **Upgrade & Migration:** No additional extension DDL or schema detection is
needed. Body policies use `df.with_http_options`; destination tables are
caller-provisioned in the workflow's target database. Existing HTTP defaults,
caller-provisioned in the workflow's origin database unless an explicit
workflow database target is supplied. Existing HTTP defaults,
signatures, grants, and previously persisted response bodies are unchanged.
- **Binary compatibility:** The new `.so` continues to work with supported older
extension schemas. Sink writes use native PostgreSQL functionality through a
Expand Down Expand Up @@ -505,9 +506,28 @@ for the separate runtime shutdown and server-side cancellation limitations.
migration is needed. Activity names, serialized inputs, and orchestration
scheduling are unchanged; the client cache is confined to activity execution.

#### SQL name-resolution hardening (#418)

- **Binary compatibility:** extension-owned runtime SQL explicitly qualifies
catalog objects, functions, types and operators rather than relying on caller
or database `search_path`. This applies when the new binary is loaded,
including with supported old schemas; no new schema detection is required.
- **Upgrade DDL:** none. Installed SQL definitions and shipped migrations are
unchanged, so no additional `ALTER EXTENSION` is needed for this hardening.
- **Replay and execution:** user-authored SQL and execution connection search
paths are preserved. Orchestration-generated row-set expansion retains its
existing SQL spelling, including casts, to preserve recorded activity inputs.
Activity names and orchestration scheduling are unchanged.
- **Coverage:** `89_schema_qualification` exercises starts, variables, monitoring
and workflow management with shadowing functions/operators and temporary types,
while checking that user SQL retains its own name resolution. The real N-1
lifecycle remains the separate replay gate.

#### Other post-tag changes

- **Dependencies (#390):** `uuid` 1.26.1 and `reqwest` 0.13.5 are binary updates, not v0.2.8 dependencies. `reqwest` uses `base64` 0.23.1, while pg_durable's direct dependency remains on 0.22.1. The `duroxide`/`duroxide-pg` pair is unchanged.
- **Dependencies (#390, #406):** `uuid` 1.26.1, `reqwest` 0.13.5 and the direct
`base64` dependency 0.23.1 are binary updates, not v0.2.8 dependencies.
The pinned `duroxide` 0.1.30 / `duroxide-pg` 0.1.34 pair is unchanged.
- **Test and release tooling (#388, #389):** shared E2E HTTP grants are restored
after lifecycle tests, and release-triggered Docker publication waits for
package assets. Neither change modifies the installed extension schema.
Expand Down
Loading