Skip to content

feat(greenfield): close database observability parity - #423

Open
mira-2026 wants to merge 11 commits into
mira/greenfield-rewrite-ops-actions-parityfrom
mira/greenfield-rewrite-database-parity
Open

feat(greenfield): close database observability parity#423
mira-2026 wants to merge 11 commits into
mira/greenfield-rewrite-ops-actions-parityfrom
mira/greenfield-rewrite-database-parity

Conversation

@mira-2026

@mira-2026 mira-2026 commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Stack

Summary

  • close database parity for the authenticated /database workflow with independent SQLite and PostgreSQL/PgBouncer fresh, retained, and unavailable states
  • discover the opted-in PgBouncer endpoint dynamically from Docker and derive the bounded PostgreSQL inventory from pg_catalog; add/remove/rename, container identity, and loopback port changes require no Dashboard config edit
  • run the privileged reconciliation inside the existing hourly cache job, with a normally NOLOGIN observer, exact activation approval, one-use catalog-bound enable token, per-application-database quarantine, and mandatory close before cache commit
  • expose bounded maintenance signals for SQLite and PostgreSQL, traffic-weighted cache/PgBouncer aggregates, ranked table health, sanitized statement metrics, and the sole named Comet/Bitmagnet torrent-count exception
  • harden SQLite snapshot capacity, restore verification, crash recovery, retention, and publication
  • add bounded immutable release/runtime retention, allocation/inode admission, physical fsync durability, crash-safe locks, and hosted browser/process build gates

Security and durability

  • only the worker-side observer password is configured; database, container, service, project, image, host, port, and application inventory are discovered rather than manually configured
  • no arbitrary SQL, raw query text, query identity, credentials, connection strings, provider output, or raw errors cross the public contract
  • observer access is limited to direct CONNECT plus exact sanitized capability functions; the isolated NOLOGIN capability owner holds the narrow statistics authority
  • application-database drift revokes observer CONNECT for that database and keeps its row visible as unavailable; cluster, control, approval, identity, or cleanup drift fails the attempt closed
  • no additional Dashboard action, schedule, polling loop, sidecar, or reconciler systemd unit is introduced
  • release/runtime publication preserves a 64 MiB and inode reserve, fsyncs files and directories before success, and safely recovers crash-left artifacts and locks

Parity

  • closes the legacy read-only database overview and /database route with reviewed security narrowing for raw query text and generic exec
  • inventory remains 113 implemented / 41 planned / 3 reviewed removals across 157 legacy endpoints
  • browser inventory remains 13 implemented / 3 planned

Verification

  • full Bun suite: 2,915 passed
  • full browser suite at the unchanged three-process policy: 841 passed
  • full coverage at the unchanged inventory/threshold: 91.45%, including exact Drizzle and Tailwind production configs
  • Storybook browser suite: 263 passed, plus production Storybook build
  • all browser, Storybook, and Bun typechecks; full lint and formatting; documentation, migration, source-boundary, and coverage-policy gates
  • real browser/process bundle builds and clean-tree release build for exact commit 51dedc979672a50153226059d99a8ca2ec6192e4
  • focused provisioning, process-supervision, privilege-drift, snapshot, retention, capacity, lock, publication, and runtime adversarial tests
  • hosted CI and automated review remain the final gates for the pushed commit

Scope

  • greenfield rewrite plus coexistence CI/Codecov integration only; no legacy application-tree changes
  • legacy frontend/backend checks now run for rewrite branches as well; the greenfield package is isolated into the runner temp root before legacy checks
  • no production deploy, database mutation, container restart, credential rotation, or PgBouncer recreation in this PR

Notes for reviewers

  • review the approval-bound prepare/enable/collect/close sequence, app-database quarantine, Docker endpoint discovery, sanitized PostgreSQL capability boundary, immutable artifact retention, and physical publication durability

@mira-2026
mira-2026 requested a review from rajohan as a code owner August 12, 2026 20:08
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown

Important

Review skipped

Too many files!

This PR contains 207 files, which is 107 over the limit of 100.

To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch.

Upgrade to a paid plan to raise the limit.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5471ffb5-5e49-4350-b707-ecdb61a81da3

📥 Commits

Reviewing files that changed from the base of the PR and between 67fe27c and 1f888a8.

⛔ Files ignored due to path filters (15)
  • bun.lock is excluded by !**/*.lock and included by **/*
  • greenfield/bun.lock is excluded by !**/*.lock and included by **/*
  • greenfield/docs/generated/configuration.md is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/packages-and-runtime.md is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/procedures.md is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/routes-and-features.md is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/automationSecurity.createPrincipal.input.schema.json is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/automationSecurity.createPrincipal.output.schema.json is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/automationSecurity.disablePrincipal.output.schema.json is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/automationSecurity.listPrincipals.output.schema.json is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/automationSecurity.replaceCapabilities.input.schema.json is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/automationSecurity.replaceCapabilities.output.schema.json is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/database.overview.input.schema.json is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/database.overview.output.schema.json is excluded by !**/generated/** and included by **/*
  • greenfield/docs/generated/schemas/securityAudit.listEvents.output.schema.json is excluded by !**/generated/** and included by **/*
📒 Files selected for processing (207)
  • .github/workflows/dashboard-checks.yml
  • .oxfmtrc.json
  • .oxlintrc.json
  • README.md
  • bunfig.toml
  • codecov.yml
  • docs/api/endpoints.md
  • docs/architecture/gateway-and-chat.md
  • docs/development/local-dev.md
  • docs/setup/new-vps.md
  • greenfield/.bun-browser-test-timings.json
  • greenfield/.bun-test-timings.json
  • greenfield/bunfig.toml
  • greenfield/codecov.yml
  • greenfield/docs/architecture/greenfield-rewrite/application-architecture.md
  • greenfield/docs/architecture/greenfield-rewrite/data-and-security.md
  • greenfield/docs/architecture/greenfield-rewrite/implementation-plan.md
  • greenfield/docs/architecture/greenfield-rewrite/progress.md
  • greenfield/docs/architecture/greenfield-rewrite/runtime-and-delivery.md
  • greenfield/docs/development/testing-and-prs.md
  • greenfield/migrations/20260804022252_dashboard-foundation/migration.sql
  • greenfield/migrations/20260804022252_dashboard-foundation/snapshot.json
  • greenfield/package.json
  • greenfield/scripts/buildSourceIdentity.test.ts
  • greenfield/scripts/buildSourceIdentity.ts
  • greenfield/scripts/checkCoverage.test.ts
  • greenfield/scripts/checkCoverage.ts
  • greenfield/scripts/checkDatabaseSchema.test.ts
  • greenfield/scripts/delivery/activateProductionRelease.ts
  • greenfield/scripts/delivery/activateProductionReleaseAdmission.test.ts
  • greenfield/scripts/delivery/buildAdmission.test.ts
  • greenfield/scripts/delivery/buildBrowser.test.ts
  • greenfield/scripts/delivery/buildRelease.test.ts
  • greenfield/scripts/delivery/buildRelease.ts
  • greenfield/scripts/delivery/databaseObservabilityProvisioning.test.ts
  • greenfield/scripts/delivery/databaseObservabilityProvisioningPolicy.ts
  • greenfield/scripts/delivery/databaseSnapshotRetention.test.ts
  • greenfield/scripts/delivery/databaseSnapshotRetention.ts
  • greenfield/scripts/delivery/deploymentLease.test.ts
  • greenfield/scripts/delivery/deploymentLease.ts
  • greenfield/scripts/delivery/exclusiveProcessLock.ts
  • greenfield/scripts/delivery/productionArtifactCapacity.ts
  • greenfield/scripts/delivery/productionArtifactRetention.test.ts
  • greenfield/scripts/delivery/productionArtifactRetention.ts
  • greenfield/scripts/delivery/productionReleaseActivation.test.ts
  • greenfield/scripts/delivery/productionReleaseActivation.ts
  • greenfield/scripts/delivery/productionReleasePublication.test.ts
  • greenfield/scripts/delivery/productionReleasePublication.ts
  • greenfield/scripts/delivery/productionRuntime.test.ts
  • greenfield/scripts/delivery/productionRuntime.ts
  • greenfield/scripts/delivery/productionRuntimePointers.ts
  • greenfield/scripts/delivery/provisioning/database-observability/README.md
  • greenfield/scripts/delivery/provisioning/database-observability/activate-observer.sql
  • greenfield/scripts/delivery/provisioning/database-observability/apply-cluster.sql
  • greenfield/scripts/delivery/provisioning/database-observability/apply-control-database-capability.sql
  • greenfield/scripts/delivery/provisioning/database-observability/apply-control-database.sql
  • greenfield/scripts/delivery/provisioning/database-observability/apply-database-access-reconciler.sql
  • greenfield/scripts/delivery/provisioning/database-observability/apply-database-capabilities.sql
  • greenfield/scripts/delivery/provisioning/database-observability/apply-reconciliation-approval.sql
  • greenfield/scripts/delivery/provisioning/database-observability/apply-torrent-view.sql
  • greenfield/scripts/delivery/provisioning/database-observability/disable-observer.sql
  • greenfield/scripts/delivery/provisioning/database-observability/enable-approved-collection.sql
  • greenfield/scripts/delivery/provisioning/database-observability/manifest.json
  • greenfield/scripts/delivery/provisioning/database-observability/prepare-approved-collection.sql
  • greenfield/scripts/delivery/provisioning/database-observability/reconcile-database-access.sql
  • greenfield/scripts/delivery/provisioning/database-observability/rollback-cluster.sql
  • greenfield/scripts/delivery/provisioning/database-observability/rollback-control-database-capability.sql
  • greenfield/scripts/delivery/provisioning/database-observability/rollback-control-database.sql
  • greenfield/scripts/delivery/provisioning/database-observability/rollback-database-access-reconciler.sql
  • greenfield/scripts/delivery/provisioning/database-observability/rollback-database-capabilities.sql
  • greenfield/scripts/delivery/provisioning/database-observability/rollback-reconciliation-approval.sql
  • greenfield/scripts/delivery/provisioning/database-observability/rollback-torrent-view.sql
  • greenfield/scripts/delivery/provisioning/database-observability/runProvisioning.ts
  • greenfield/scripts/delivery/provisioning/database-observability/verify-cluster.sql
  • greenfield/scripts/delivery/provisioning/database-observability/verify-control-database-capability.sql
  • greenfield/scripts/delivery/provisioning/database-observability/verify-control-database.sql
  • greenfield/scripts/delivery/provisioning/database-observability/verify-database-access-reconciler.sql
  • greenfield/scripts/delivery/provisioning/database-observability/verify-database-capabilities.sql
  • greenfield/scripts/delivery/provisioning/database-observability/verify-database.sql
  • greenfield/scripts/delivery/provisioning/database-observability/verify-reconciliation-approval.sql
  • greenfield/scripts/delivery/provisioning/database-observability/verify-torrent-view.sql
  • greenfield/scripts/delivery/releaseIdentity.test.ts
  • greenfield/scripts/delivery/releaseIdentity.ts
  • greenfield/scripts/delivery/releaseStaging.ts
  • greenfield/scripts/delivery/systemdProductionServices.test.ts
  • greenfield/scripts/development/developmentFrontend.test.ts
  • greenfield/scripts/development/developmentRemoteProxy.ts
  • greenfield/scripts/developmentFrontend.ts
  • greenfield/scripts/documentation/artifacts.test.ts
  • greenfield/scripts/documentation/files.test.ts
  • greenfield/scripts/documentation/jsonSchema.test.ts
  • greenfield/scripts/documentation/jsonSchema.ts
  • greenfield/scripts/files/boundedFile.ts
  • greenfield/scripts/generateDocs.test.ts
  • greenfield/scripts/generateDocs.ts
  • greenfield/scripts/runCoverage.test.ts
  • greenfield/scripts/runCoverage.ts
  • greenfield/scripts/sourceBoundaries/policy.test.ts
  • greenfield/scripts/sourceBoundaries/sourceTopologyPolicy.ts
  • greenfield/scripts/testSupport/productionDeliveryFixture.ts
  • greenfield/src/app/dashboardServer.test.ts
  • greenfield/src/app/dashboardServer.ts
  • greenfield/src/app/databaseMaintenance.test.ts
  • greenfield/src/app/databaseMaintenance.ts
  • greenfield/src/app/developmentWorker.ts
  • greenfield/src/app/server.ts
  • greenfield/src/app/trpcHttpHandler.ts
  • greenfield/src/app/worker.test.ts
  • greenfield/src/app/worker.ts
  • greenfield/src/browser/api/trpcClient.ts
  • greenfield/src/browser/auth/LoginRoute.test.tsx
  • greenfield/src/browser/database/DatabaseRoute.test.tsx
  • greenfield/src/browser/database/DatabaseRoute.tsx
  • greenfield/src/browser/database/PostgresqlDatabaseOverview.tsx
  • greenfield/src/browser/database/PostgresqlDatabaseTables.tsx
  • greenfield/src/browser/database/databaseQueries.ts
  • greenfield/src/browser/database/databaseRouteSearch.test.ts
  • greenfield/src/browser/database/databaseRouteSearch.ts
  • greenfield/src/browser/jobs/JobsRoute.test.tsx
  • greenfield/src/browser/layout/DashboardShell.test.ts
  • greenfield/src/browser/layout/DashboardShell.tsx
  • greenfield/src/browser/layout/dashboardNavigation.ts
  • greenfield/src/browser/monitoring/MonitoringRoutes.test.tsx
  • greenfield/src/browser/notifications/NotificationCenter.test.tsx
  • greenfield/src/browser/router.tsx
  • greenfield/src/browser/routes/database.lazy.test.tsx
  • greenfield/src/browser/routes/database.lazy.tsx
  • greenfield/src/browser/sessions/GatewaySessionsBrowser.test.tsx
  • greenfield/src/browser/sessions/GatewaySessionsView.test.tsx
  • greenfield/src/browser/ui/Alert.test.tsx
  • greenfield/src/browser/ui/Alert.tsx
  • greenfield/src/browser/ui/stories/Alert.stories.tsx
  • greenfield/src/contracts/contractRegistry.ts
  • greenfield/src/contracts/database.test.ts
  • greenfield/src/contracts/database.ts
  • greenfield/src/contracts/databaseObservabilityCollector.ts
  • greenfield/src/contracts/security.test.ts
  • greenfield/src/contracts/security.ts
  • greenfield/src/server/database/migrations/migrationGraph.test.ts
  • greenfield/src/server/database/runtime/databasePath.ts
  • greenfield/src/server/database/runtime/databaseService.test.ts
  • greenfield/src/server/database/runtime/databaseService.ts
  • greenfield/src/server/database/runtime/databaseSnapshot.test.ts
  • greenfield/src/server/database/runtime/databaseSnapshot.ts
  • greenfield/src/server/database/schema/automationPrincipalCapabilities.ts
  • greenfield/src/server/domains/cache/heartbeatProjection.test.ts
  • greenfield/src/server/domains/cache/providerRegistry.ts
  • greenfield/src/server/domains/cache/service.test.ts
  • greenfield/src/server/domains/cache/service.ts
  • greenfield/src/server/domains/chat/service.test.ts
  • greenfield/src/server/domains/chat/service.ts
  • greenfield/src/server/domains/database/procedures.test.ts
  • greenfield/src/server/domains/database/procedures.ts
  • greenfield/src/server/domains/database/routes.ts
  • greenfield/src/server/domains/database/service.test.ts
  • greenfield/src/server/domains/database/service.ts
  • greenfield/src/server/domains/database/snapshotRepository.test.ts
  • greenfield/src/server/domains/database/snapshotRepository.ts
  • greenfield/src/server/domains/database/sqliteLifecycle.test.ts
  • greenfield/src/server/domains/database/sqliteLifecycle.ts
  • greenfield/src/server/domains/jobs/actionExecutors.test.ts
  • greenfield/src/server/domains/jobs/actionExecutors.ts
  • greenfield/src/server/domains/jobs/actionRegistry.test.ts
  • greenfield/src/server/domains/jobs/actionRegistry.ts
  • greenfield/src/server/domains/jobs/coordinator.test.ts
  • greenfield/src/server/domains/jobs/coordinator.ts
  • greenfield/src/server/domains/jobs/repository.ts
  • greenfield/src/server/domains/jobs/workerRuntime.test.ts
  • greenfield/src/server/domains/jobs/workerRuntime.ts
  • greenfield/src/server/platform/configuration/configurationRegistry.test.ts
  • greenfield/src/server/platform/configuration/databaseObservabilityConfiguration.ts
  • greenfield/src/server/platform/configuration/workerConfiguration.test.ts
  • greenfield/src/server/platform/configuration/workerConfiguration.ts
  • greenfield/src/server/platform/runtime/applicationRuntime.test.ts
  • greenfield/src/server/platform/runtime/applicationRuntime.ts
  • greenfield/src/server/test/support/freshDatabase.test.ts
  • greenfield/src/server/test/support/freshDatabase.ts
  • greenfield/src/server/test/support/requestContext.ts
  • greenfield/src/server/trpc/appRouter.ts
  • greenfield/src/server/trpc/context.test.ts
  • greenfield/src/server/trpc/context.ts
  • greenfield/src/server/trpc/procedureErrorPolicy.ts
  • greenfield/src/shared/browserRouteRegistry.ts
  • greenfield/src/shared/configuration/applicationConfigurationRegistry.ts
  • greenfield/src/shared/databaseMigrationManifest.ts
  • greenfield/src/shared/databaseObservabilityPolicy.ts
  • greenfield/src/shared/databaseObservabilityReconciliation.ts
  • greenfield/src/shared/databaseSnapshotManifest.ts
  • greenfield/src/test/integration/delivery/productionReleaseLifecycle.test.ts
  • greenfield/src/test/parity/fixtures/frontend-routes.json
  • greenfield/src/test/parity/fixtures/greenfield-contracts.json
  • greenfield/src/test/parity/fixtures/legacy-endpoints.json
  • greenfield/src/test/parity/parityInventory.test.ts
  • greenfield/src/test/parity/parityInventorySchemas.ts
  • greenfield/src/worker/database/bunSqlDatabaseObservabilityCollector.test.ts
  • greenfield/src/worker/database/bunSqlDatabaseObservabilityCollector.ts
  • greenfield/src/worker/database/dockerDatabaseObservabilityEndpointResolver.test.ts
  • greenfield/src/worker/database/dockerDatabaseObservabilityEndpointResolver.ts
  • greenfield/src/worker/database/fixedDatabaseObservabilityReconciler.test.ts
  • greenfield/src/worker/database/fixedDatabaseObservabilityReconciler.ts
  • greenfield/src/worker/database/fixedSqliteLifecycleMaintenance.test.ts
  • greenfield/src/worker/database/fixedSqliteLifecycleMaintenance.ts
  • greenfield/systemd/mira-dashboard-web.service
  • greenfield/systemd/mira-dashboard-worker.service
  • package.json
  • scripts/buildFrontend.ts
  • scripts/frontendBuild.ts

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • Review on demand using usage pricing

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mira-2026 mira-2026 changed the title mira/greenfield rewrite database parity feat(greenfield): close database observability parity Aug 12, 2026
@mira-2026 mira-2026 added status: needs-review Ready for human or agent review type: feature Adds user-visible functionality type: tests Adds or updates automated tests/coverage type: security Security, auth, secrets, or trust-boundary work type: documentation Documentation, README, comments, and operational notes area: database Database dashboard, Postgres, PgBouncer, or query views area: frontend Frontend UI, client state, routing, and browser behavior area: backend Backend API, server routes, services, and integrations area: ops Operational actions, deploys, services, backups, and health labels Aug 12, 2026
@rajohan

rajohan commented Aug 12, 2026

Copy link
Copy Markdown
Owner

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

@mira-2026 mira-2026 added type: performance Performance, resource usage, or responsiveness work area: ci CI, test runners, workflow checks, and release gates labels Aug 12, 2026
@rajohan

rajohan commented Aug 12, 2026

Copy link
Copy Markdown
Owner

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6585efc7a6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread greenfield/src/server/database/runtime/databaseSnapshot.ts Outdated
Comment thread greenfield/src/server/domains/jobs/actionRegistry.ts Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1f888a81bd

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread greenfield/scripts/runCoverage.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: backend Backend API, server routes, services, and integrations area: ci CI, test runners, workflow checks, and release gates area: database Database dashboard, Postgres, PgBouncer, or query views area: frontend Frontend UI, client state, routing, and browser behavior area: ops Operational actions, deploys, services, backups, and health status: needs-review Ready for human or agent review type: documentation Documentation, README, comments, and operational notes type: feature Adds user-visible functionality type: performance Performance, resource usage, or responsiveness work type: security Security, auth, secrets, or trust-boundary work type: tests Adds or updates automated tests/coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants