Skip to content

fix(deploy): release a settled idempotency key so redeploys aren't permanently blocked#5908

Closed
TheodoreSpeaks wants to merge 1 commit into
stagingfrom
fix/deploy-idempotency-key-release
Closed

fix(deploy): release a settled idempotency key so redeploys aren't permanently blocked#5908
TheodoreSpeaks wants to merge 1 commit into
stagingfrom
fix/deploy-idempotency-key-release

Conversation

@TheodoreSpeaks

Copy link
Copy Markdown
Collaborator

Symptom

Redeploying a workflow fails forever with:

Idempotency key was already used for a different deployment request

It is not recoverable from the workflow — editing the workflow changes the request hash, which is precisely what keeps triggering it.

Root cause

The deploy orchestrator scopes one idempotency key to a whole request (orchestration/deploy.ts: idempotencyKey: params.requestId), while requestHash is derived from workflow state.

That holds when one request performs one deployment. The agent path breaks it: a single /api/mothership/chat request deploys, edits, then redeploys. The second deployment presents the same key with a new hash, so prepareWorkflowDeployment took the conflict branch — and kept taking it, because the key was never released.

Two defects in the same block:

  1. Ordering — the request-hash comparison ran before the terminal-status check, making the release path unreachable whenever the hash differed. Its own comment ("a terminally failed or superseded attempt releases its idempotency key") never applied to the case that needed it.
  2. active wasn't settled — a successfully completed operation reserved its key permanently.

Evidence (staging)

One chat request 58a47006 deployed three workflows:

workflow status created
870c86b7… active 00:02:31
4c7600a2… active 00:06:01
c4f8bd85… active 00:06:02

It then tried to redeploy 870c86b7 — same key, new hash — and failed five times (00:07:18 → 00:08:57), each toolName: "redeploy"idempotency_conflict. The held row was active, so the key was never freed.

Fix

Release the key whenever the prior attempt is settled (active, failed, or superseded) and the request genuinely differs. Conflict is now raised only when a different deployment is still in flight (pending / preparing / activating) — the genuinely ambiguous case idempotency exists to guard.

Duplicate-submission protection is unchanged: same key + same hash still reuses the existing operation and never redeploys.

key hash prior status before after
same same in-flight / active reuse reuse (unchanged)
same same failed / superseded release + retry release + retry (unchanged)
same diff in-flight conflict conflict (unchanged)
same diff active ❌ conflict forever ✅ release + deploy
same diff failed / superseded ❌ conflict forever ✅ release + deploy

Checks

  • bunx vitest run lib/workflows — 989 passed (54 files); 4 new cases, all 11 pre-existing deployment-operations tests unchanged and green
  • bun run type-check — no new errors (7 pre-existing: better-auth + @daytonaio/sdk)
  • bun run lint:check — clean

🤖 Generated with Claude Code

https://claude.ai/code/session_018asmKsWQ5Vi7T7wD9uHofz

…rmanently blocked

Redeploying a workflow could fail forever with "Idempotency key was already used
for a different deployment request", with no way to clear it from the workflow.

The deploy orchestrator scopes one idempotency key to a whole request
(`idempotencyKey: params.requestId`) while the request hash is derived from
workflow state. That is fine when one request performs one deployment, but the
agent path issues several deployments per chat request: deploy, edit, redeploy.
The second deployment presents the same key with a new hash, so
prepareWorkflowDeployment took the conflict branch — and kept taking it, because
the key was never released.

Two problems, both in the same block:

- The request-hash comparison ran BEFORE the terminal-status check, so the
  release path was unreachable whenever the hash differed. Its own comment
  ("a terminally failed or superseded attempt releases its idempotency key")
  therefore never applied to the case that needed it.
- A completed (`active`) operation was not treated as settled at all, so a
  successful deployment reserved its key permanently.

Reordered so the key is released whenever the prior attempt is settled
(`active`, `failed`, or `superseded`) and the request genuinely differs. A
conflict is now raised only when a *different* deployment is still in flight
(`pending`/`preparing`/`activating`), which is the ambiguous case idempotency
is meant to guard. True duplicate submissions (same key, same hash) still reuse
the existing operation and never redeploy.

Observed on staging: one chat request (58a47006) deployed three workflows, then
its redeploy of the first failed five times in a row with this error.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018asmKsWQ5Vi7T7wD9uHofz
@vercel

vercel Bot commented Jul 24, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
docs Skipped Skipped Jul 24, 2026 12:18am

Request Review

@cursor

cursor Bot commented Jul 24, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Changes deployment idempotency semantics in prepareWorkflowDeployment, which can affect concurrent deploy/redeploy behavior but preserves duplicate-submission reuse and still blocks ambiguous in-flight conflicts.

Overview
Fixes permanent idempotency_conflict when the same request id is reused with a new deployment payload (e.g. agent chat redeploying an edited workflow while idempotencyKey stays requestId).

prepareWorkflowDeployment now treats active as settled alongside failed / superseded, and reorders idempotency handling: same key + same hash still reuses the existing operation; same key + different hash conflicts only while work is in flight (preparing / activating); otherwise it clears the key on the old row and creates a fresh deployment.

Adds four tests covering active/failed release with a different hash, unchanged conflict during activation, and duplicate reuse for completed active operations.

Reviewed by Cursor Bugbot for commit af46917. Bugbot is set up for automated code reviews on this repo. Configure here.

@greptile-apps

greptile-apps Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR corrects deployment idempotency handling for request-scoped keys.

  • Reuses an existing operation when the key and request hash both match.
  • Releases the key for a different request only after the prior operation reaches active, failed, or superseded.
  • Continues rejecting different requests while the prior operation is preparing or activating.
  • Adds regression coverage for active and failed key release, in-flight conflicts, and active duplicate reuse.

Confidence Score: 5/5

The PR appears safe to merge, with settled and in-flight deployment states handled consistently by the revised idempotency logic.

The workflow transaction serializes preparation, terminal statuses align with the new settled classification, same-hash duplicates remain reusable, and different in-flight requests still return a conflict.

Important Files Changed

Filename Overview
apps/sim/lib/workflows/persistence/deployment-operations.ts Reorders hash and lifecycle checks so settled operations can release request-scoped idempotency keys while in-flight operations remain protected.
apps/sim/lib/workflows/persistence/deployment-operations.test.ts Adds focused regression tests covering settled-key reassignment, in-flight conflict behavior, and genuine duplicate reuse.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Existing operation for idempotency key] --> B{Same request hash?}
  B -->|Yes| C{Failed or superseded?}
  C -->|No| D[Reuse existing operation]
  C -->|Yes| E[Release key and create fresh operation]
  B -->|No| F{Active, failed, or superseded?}
  F -->|No| G[Return idempotency conflict]
  F -->|Yes| E
Loading

Reviews (1): Last reviewed commit: "fix(deploy): release a settled idempoten..." | Re-trigger Greptile

@waleedlatif1
waleedlatif1 deleted the fix/deploy-idempotency-key-release branch July 25, 2026 00:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant