Repository navigation
Register transactional step factories' checkpoint stores with DBOS - #607
Merged
Merged
Conversation
Transactional step factories keep their checkpoints in tx_step_outputs in the application database, out of DBOS's reach. Each factory now registers a store with DBOS that deletes a workflow's checkpoints from a step on, so DBOS can clear them when it discards a workflow's history (rewind, and later deleteWorkflows, retention and completion cleanup). - Factories register when they are created; registering after DBOS.launch() throws, and registering the same store twice keeps one entry. - Each factory deletes through its own library's transaction (JDBC, Jdbi, jOOQ, Spring), so the delete commits whatever the pool's autocommit setting. - TxStepSchema.createTable commits when autocommit is off; a pool configured that way used to roll the CREATE TABLE back.
StepCheckpointStore gains deleteCheckpoints(Collection<String>), which deletes every checkpoint of several workflows in one statement (workflow_id = ANY(?)), for deleteWorkflows and retention. With two methods it is no longer a functional interface, so each factory registers a private nested implementation instead of a method reference; the interface's methods do not become public methods of the factories. The batch delete goes through each library's own transaction, as the single-workflow delete does, and skips an empty collection.
- Jdbi deletes run through handle.begin()/commit(). useTransaction does not commit a connection that arrives with autocommit off: Jdbi takes it for a transaction already in progress, and the pool then rolls the delete back. - The jOOQ batch delete binds the ID array on the JDBC connection, since jOOQ cannot bind arrays when the DSLContext has no dialect set. - deleteStepCheckpoints wraps unchecked store failures too, so a Jdbi or jOOQ error names the workflow; a failed batch names its first few IDs only. - Document that factories must be created before DBOS.launch().
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Registration can race with launch and leave a successfully created factory absent from the executor snapshot.
Review effort: Balanced
Findings: 1
What changed in this PR
Adds a registry enabling DBOS to delete transactional-step checkpoints across supported database integrations.
Changes:
- Registers checkpoint stores before launch and exposes deletion helpers.
- Adds committed deletion implementations for JDBC, Jdbi, jOOQ, and Spring.
- Adds autocommit and registration coverage.
| File | Description |
|---|---|
transact/.../StepCheckpointRegistryTest.java |
Tests registry and deletion behavior. |
transact/.../TxStepSchema.java |
Adds checkpoint deletion SQL and DDL commit. |
transact/.../PostgresStepFactory.java |
Registers a default checkpoint store. |
transact/.../JdbcStepFactory.java |
Implements JDBC checkpoint deletion. |
transact/.../StepCheckpointStore.java |
Defines the internal store contract. |
transact/.../DBOSIntegration.java |
Exposes store registration. |
transact/.../DBOSExecutor.java |
Adds registry-backed deletion helpers. |
transact/.../DBOS.java |
Collects stores and snapshots them at launch. |
transact-spring.../TransactionalStepJdbcIntegrationTest.java |
Tests Spring deletion integration. |
transact-spring.../DBOSTestAccess.java |
Exposes executor to Spring tests. |
transact-spring.../TransactionalStepFactory.java |
Registers and deletes Spring checkpoints. |
transact-jooq.../JooqStepFactoryTest.java |
Tests jOOQ deletion. |
transact-jooq.../JooqStepFactoryNoAutoCommitTest.java |
Tests jOOQ explicit commits. |
transact-jooq.../DBOSTestAccess.java |
Exposes executor to jOOQ tests. |
transact-jooq.../JooqStepFactory.java |
Implements jOOQ checkpoint deletion. |
transact-jdbi.../JdbiStepFactoryTest.java |
Tests Jdbi deletion. |
transact-jdbi.../JdbiStepFactoryNoAutoCommitTest.java |
Tests Jdbi explicit commits. |
transact-jdbi.../DBOSTestAccess.java |
Exposes executor to Jdbi tests. |
transact-jdbi.../JdbiStepFactory.java |
Implements Jdbi checkpoint deletion. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
kraftp
approved these changes
Oct 7, 2026
devhawk
added a commit
that referenced
this pull request
Oct 7, 2026
Closes #537. Rewind re-runs a **terminal** workflow in place, under the same ID, from a start step. Steps before the cut replay from their checkpoints; everything from the cut on is discarded and runs again. A workflow that is still PENDING, ENQUEUED or DELAYED is refused ("cancel it first"). Python, TypeScript and Go already have it; this follows Python and TypeScript, including clearing the legacy `output`/`error` columns. ## What a rewind does In one system-database transaction, from the start step on: - rolls each event back to its last value published before the cut, from `workflow_events_history`, then deletes those history rows; - deletes the step checkpoints in `operation_outputs`; - deletes the streams' close sentinels so the replay can append again. Stream entries are kept, since their offsets are what readers resume from; - deletes the messages consumed by discarded steps, and any not yet consumed; - deletes the `workflow_output` row and clears the legacy columns; - re-enqueues the workflow (internal queue, or the given queue and partition), resetting recovery attempts, deadline, dedup ID and timestamps and clearing `owner_xid`; the application version is restamped only when given; - re-checks the status it read, failing with "retry the rewind" if it changed. Before that, `DBOS.rewindWorkflow` deletes the transactional step factories' `tx_step_outputs` rows from the cut on, through the checkpoint-store registry from #607. If a delete fails, the workflow keeps its terminal status and is not re-enqueued, and the rewind can be retried. Called from a workflow, the rewind is a step (`DBOS.rewindWorkflow`), so a recovered caller does not rewind again once that step is recorded. Like the other built-in steps that write, it runs again on recovery if the process dies after the rewind commits and before its checkpoint is written (#591). ## API - `DBOS.rewindWorkflow(workflowId, startStep[, RewindOptions])` and the same on `DBOSClient`. `RewindOptions` carries the queue, partition key and application version. `startStep` 0 re-runs the whole workflow. - A client cannot reach the application databases, so `DBOSClient.rewindWorkflow` leaves the step factories' checkpoints in place: a transactional step past the cut replays its recorded result. The javadoc says so and points to rewinding from the application. ## Migration 121 Adds `notifications.consumed_by_function_id` as `INT4` (`INTEGER` would be 8 bytes on CockroachDB), and `recv` now records the consuming step there, so a rewind can delete exactly the messages consumed past the cut. A message consumed before 121 (or by a version that doesn't record the step) can't be matched and stays consumed; the javadoc says so. ## Conductor Answers `rewind_workflow` (`workflow_id`, `start_step`, `application_version`, `queue_name`, `queue_partition_key`; a missing `start_step` means the whole history). ## Known limitation The step-factory deletes run after an unlocked status check and before the system-database transaction, as in the other SDKs. A concurrent rewind or resume landing in between can delete checkpoints a replay has just written. The window is narrow and the fix is being settled across SDKs. ## Tests `RewindTest` covers events, streams, messages (including one consumed without a recorded step, as before migration 121), legacy outcome columns, step-factory checkpoints (including a failed delete and an active workflow), the client, in-workflow and in-step calls with recovery, and refusal of active and missing workflows. Jdbi, jOOQ and Spring tests check that a rewind re-runs the transaction. `ConductorTest` covers the message with and without a start step; `MigrationManagerTest` covers 121, including a database stopped at 120. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Transactional step factories keep their checkpoints in
tx_step_outputsin the application database, and DBOS has no way to reach them: each factory is built on its own datasource and DBOS never learns it exists. Workflow rewind (#537), cleaning up afterdeleteWorkflowsand retention (#601), and deleting checkpoints when a workflow completes all need DBOS to delete a workflow's rows there. This PR adds the registry they share. It has no user-visible feature of its own; nothing in production calls the delete helpers yet.What's added
StepCheckpointStore(internal): delete one workflow's checkpoints from a step on (for rewind), and delete every checkpoint of several workflows in one statement (workflow_id = ANY(?), for deletes and retention). Each commits before returning.DBOSIntegration.registerStepCheckpointStore.DBOSExecutor.deleteStepCheckpoints(...)calls every registered store; a failure names the workflow (a batch names its first few IDs).PostgresStepFactory(JDBC, Jdbi, jOOQ) in its constructor, Spring'sTransactionalStepFactoryininitialize(). Each registers a private nested store, so the interface's methods don't become public methods of the factories. Registering the same store twice keeps one entry.handle.begin()/commit(), jOOQdsl.transaction(binding the ID array on the JDBC connection, which works without a dialect), Spring a fresh connection with an explicit commit, asrecordErrordoes.PostgresStepFactory's delete methods are protected with an explicit-commit default, so existing subclasses keep compiling.Behaviour changes
DBOS.launch(). The constructor (and Spring'sinitialize()) now throwsIllegalStateExceptionafter launch, as Python does for datasources: DBOS needs every factory when it launches. Spring's registrar already runs before launch, and nothing in the repo, examples or docs builds a factory after launch, but it belongs in the release notes.TxStepSchema.createTablecommits when autocommit is off. On a pool configured without autocommit theCREATE TABLEused to be rolled back when the connection went back to the pool.Tests
StepCheckpointRegistryTest: deletes from a step and by batch on a pool without autocommit, the default delete committing, an empty batch not touching the database, registration refused after launch, a store registered twice, and failures (checked and unchecked) naming the workflow.JdbiStepFactoryNoAutoCommitTestandJooqStepFactoryNoAutoCommitTest(no dialect) failed before the fixes in the last commit.Not in this PR: Jdbi steps have the same autocommit-off problem (
inTransactionnever commits, so the step's writes and checkpoint are rolled back while DBOS records the step as done). That's a separate fix.🤖 Generated with Claude Code