Skip to content

StartFlow: backend 10 s RPC cap hides the caller deadline and loses admission acknowledgements #755

Closed as not planned

Description

@longquanzheng

Observed

An application's StartFlow call against dexcli dev (Dex Server server/v1.14.2) failed with an error after 9.9 s although the Flow had been admitted and later completed normally. The caller therefore cannot tell whether the start was accepted.

  • FlowType TextSummaryFlow, FlowID text-summary-6eeedd12-c744-4edf-94dc-23106b0f20b9, diagnostic RunID 01a16592-747d-799f-818e-831c0a4f3318.
  • Admitted 2026-10-22T21:44:19.325Z, completed 21:44:24.833Z. Backend log: the start RPC took 9.900442073 s and ended at 21:44:20.847Z; the application handler returned 500 at 21:44:20.851Z.
  • The application's own handler context was 30 s; the start was issued with a stable FlowID, complete RequestID, IDReuseDisallow and same-request AlreadyStarted IgnoreError.
  • Environment: a CPU-limited container (1 CPU; 941 of 6026 CFS periods throttled) running the local engine. Raising the limit to 2 CPU removed the symptom, but the timeout boundary below remains.

Code locations

  • server/service/api/service.go:282 passes the request context to StartInterpreterWorkflow; server/service/client/temporal/client.go:310 calls ExecuteWorkflow.
  • Pinned backend SDK github.com/superdurable/temporal-sdk-go v1.47.1-superdurable.3: internal/internal_utils.go:27-32 sets both the default and the maximum RPC timeout to 10 s, and lines 111-124 clamp a longer parent deadline to that maximum; internal/internal_workflow_client.go:2185-2191 builds the start RPC context.
  • Application SDK v1.14.1 dex/client.go:422 forwards the caller's context unchanged.

So a caller's longer deadline is silently reduced to 10 s for the start RPC, and a slow but successful admission is reported as a plain error.

Expected

  1. The start RPC honours the caller's deadline (or a documented, configurable server limit), instead of a hidden 10 s cap.
  2. When the outcome is uncertain (deadline exceeded after the request was sent), the error is distinguishable from a rejection, and the SDK documents or performs a bounded replay of the identical start (same FlowID, RequestID, input, reuse policy and IgnoreError) that confirms admission without a second business effect, also after the Flow has completed.

Hypothesis (unverified)

The 10 s maximum comes from the upstream SDK default and was not meant to cap server-side starts. Raising or plumbing the maximum is enough for (1); (2) needs a typed uncertain-outcome error in the server API and a replay helper in the Go SDK.

Acceptance

  • A test against a real Dex Server and backend that delays admission beyond 10 s but within the caller's deadline returns success.
  • A start that exceeds the caller's deadline returns a typed uncertain-outcome error, and the documented replay returns the accepted execution without starting a second one.

Activity

  1. added
    type:bugWrong behavior in released or main code
    P2Normal priority
    codex-readySelf-contained; can be handed to a Codex worker as is
    on Oct 26, 2026
  2. longquanzheng commented on Nov 9, 2026

    @longquanzheng
    ContributorAuthor

    The 10 s cap was in the Temporal client used by the removed Go Server (#1059). server-zd honours the caller's deadline. Closing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal prioritycodex-readySelf-contained; can be handed to a Codex worker as issize:MA few daystype:bugWrong behavior in released or main code

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions