Skip to content

Port system-database migrations 115-120, and require the last migration - #606

Merged
devhawk merged 3 commits into
mainfrom
migrations-115-120
Oct 6, 2026
Merged

devhawk merged 3 commits into
mainfrom
migrations-115-120

Conversation

@devhawk

@devhawk devhawk commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

Python, TypeScript and Go rebuilt three hot system-database indexes in migrations 115-120 (py#857, ts#1367, go#476). Java stopped at 114. The migration version is one counter that every SDK shares, so a database first migrated by Java never got these indexes, even when other SDKs' apps share it.

Migrations 115-120. The DDL is copied verbatim from Python, at the same numbers:

Creates Drops
115 / 116 idx_workflow_status_in_flight_v2 idx_workflow_status_in_flight
117 / 118 idx_workflow_status_partition_dequeue_v3 idx_workflow_status_partition_dequeue_v2
119 / 120 idx_operation_outputs_completed_at_function_name_v2 idx_operation_outputs_completed_at_function_name

Each new index carries application_name as an INCLUDE column, so app-scoped counts and dequeues run index-only. All six run online (CONCURRENTLY, outside a transaction) on Postgres and transactionally on CockroachDB, like the other index migrations. On a Postgres copy of the schema, EXPLAIN shows the in-flight count, the dequeue, the partitioned head probe and loose scan, and the step aggregates each using the matching new index.

Required schema version. MigrationManager.MINIMUM_SYSDB_VERSION, a hand-kept floor of 111, is replaced by latestMigrationVersion(), the SDK's last migration (now 120). An executor with migrate disabled, and every DBOSClient, now requires every migration this SDK defines. A schema migrated ahead by a newer SDK is still accepted.

This removes a public constant. The next release adds migrations that new code depends on (rewind, starting at 121), so the floor would have had to rise to the last migration then anyway.

Tests. A new test migrates a database to 114 and then to 120, and checks that the old indexes are dropped, the new ones exist with application_name as their only INCLUDE column (on CockroachDB, where it is reported as STORING), and a re-run is a no-op. Others cover a schema ahead of the SDK and the migration count being the same in every listen/notify and CockroachDB configuration. MigrationManagerTest and ClientSysDbVersionTest pass locally on Postgres and CockroachDB.

Fixes #573

🤖 Generated with Claude Code

Python, TypeScript and Go rebuilt three hot system-database indexes in
migrations 115-120: better key order, application_name as an INCLUDE
column so app-scoped counts and dequeues run index-only, and the
superseded indexes dropped. The migration version is one counter that
every SDK shares, so a database first migrated by Java stopped at 114
and never got these indexes, even when other SDKs' apps share it.

The DDL is copied verbatim from Python, at the same migration numbers.
All six run online (CONCURRENTLY, outside a transaction) on Postgres and
transactionally on CockroachDB, like the other index migrations. They
only rebuild indexes, so MINIMUM_SYSDB_VERSION stays at 111.

Fixes #573
MINIMUM_SYSDB_VERSION was a hand-maintained floor below the last
migration, kept low by reasoning about which later migrations each query
could do without. Replace it with latestMigrationVersion(), the length of
the migration list: an executor with migrate disabled, and a DBOSClient,
now require every migration this SDK defines. A schema migrated ahead by
a newer SDK is still accepted.
…oachDB

latestMigrationVersion() rebuilt and formatted the whole migration list
on every call to take its size. Compute it once. A test checks that the
count is the same in every listen/notify and CockroachDB configuration.

Comment migrations 115-120 with what each one creates or drops; 119's
comment named an unrelated index. The 115-120 test now also checks the
INCLUDE column on CockroachDB, where it is reported as STORING.
@devhawk
devhawk requested review from kraftp, maxdml and qianl15 and a balanced review from Copilot October 5, 2026 23:20

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The migrations, validation changes, and cross-database tests consistently implement the stated requirements.

Review effort: Balanced
Findings: None

What changed in this PR

Ports shared system-database migrations 115–120 and requires schemas to reach the SDK’s latest migration.

Changes:

  • Rebuilds three indexes with application_name included.
  • Uses the latest migration as the required schema version.
  • Adds PostgreSQL and CockroachDB migration coverage.
File Description
MigrationManager.java Adds migrations 115–120 and latest-version validation.
DBOSClient.java Updates schema-version documentation.
DBOS.java Clarifies migration requirements.
QueuesDAO.java Updates index references in documentation.
MigrationManagerTest.java Tests index upgrades and version validation.
ClientSysDbVersionTest.java Tests client enforcement of the latest version.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@devhawk
devhawk merged commit 55b383a into main Oct 6, 2026
13 checks passed
@devhawk
devhawk deleted the migrations-115-120 branch October 6, 2026 15:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add migrations 115–120: rebuild three indexes with INCLUDE (application_name)

3 participants