fix(sdk): agent-loop error recovery, lossless compaction, and loop/wake hardening

[matej21]

Fixes the three high-severity findings from the agent-SDK architecture review (CORR-1..3), plus the two related mediums they subsume (CORR-6 crash-window message loss, CORR-10 lost manual pause).

Context

Git history shows this is one bug family patched case-by-case: 1cb6a25 introduced preserve-for-retry token semantics, then b2cfd17, c2220d6 and 245b2b1 each fixed one instance of the resulting resume_from_error ↔ infer loop (empty-response, aborted, non-retryable). This PR closes the family with three invariants instead of a fifth spot patch.

Invariant 1 — error-resume cycles back off (CORR-1)

A retryable LLM error (rate limit / server error / network / timeout) that outlives withLLMRetry's 5 attempts keeps the dequeue token unconsumed, so decide() re-entered infer immediately: an unbounded, backoff-free spin that pinned the loop and sent the parent one mailbox error message per cycle during a provider outage.

• AgentState now tracks consecutiveInferenceFailures / lastInferenceFailureAt (reducers: inference_failed increments, inference_completed resets, session_restarted resets the episode).
• continue()'s resume_from_error branch waits out an exponential backoff (5s base, 60s cap, per-agent errorResumeBackoff config) and yields instead of spinning; a timer re-schedules processing when the backoff elapses.
• onError hooks (parent notification) fire once per error episode, not once per cycle.
• Capped rather than ceilinged: the preserved message keeps retrying at the max delay until the provider recovers — dropping it would lose user work.

Invariant 2 — tokens are consumed only when the turn commits (CORR-2, CORR-6)

markConsumed ran before executeAfterInference, so a plugin returning {action:'retry'} recursed into runInference with the mailbox message already consumed: a pure-message turn stranded the agent in inferring (→ idle, unrecoverable), a tool-result turn double-appended pendingMessages and produced duplicate tool_result blocks that Anthropic rejects.

• Consumption moved to the commit points, after the retry decision is final.
• New inference_retried event rolls back the staged turn (clears pendingMessages, like the inference_failed rollback from 1cb6a25) so the retry re-collects the same unconsumed messages — nothing lost, nothing duplicated.
• Ordering is commit-then-consume: a crash in the gap now re-delivers the message on restart (at-least-once) instead of silently dropping it (session_restarted wipes staged pendingMessages, so consume-first lost the message).

Invariant 3 — pause is never clobbered mid-turn (CORR-3, CORR-10)

The tool_exec loop ran all pending tools without re-checking status, and the tool_completed/tool_failed/inference_completed reducers recomputed status unconditionally — a pause from a tool hook (or a manual pauseAgent() landing mid-inference) was silently overwritten, the paused tool re-executed its side effects, and the history could end with an unpaired tool_use.

• Reducers preserve paused; the tool_exec loop stops at the first pause (remaining tools stay pending and run after resume).
• afterToolCall pause still emits the executed tool's result event — side effects already ran, so the committed tool_use gets its tool_result and nothing re-executes on resume. (beforeToolCall pause keeps gate semantics: tool never started, runs after resume.)
• agent_resumed routes back to tool_exec while tool calls are pending — resuming into pending would have run inference with unresolved tool calls.

Tests

New error-recovery.integration.test.ts (6 tests): outage → backoff timing between failure/resume events, single parent notification, message survives and is delivered exactly once; afterInference retry for pure-message and tool-result turns (no stranding, no duplicate tool results in history or LLM request); beforeToolCall/afterToolCall pause (nothing runs while paused, exactly-once execution across resume); manual pause during in-flight inference (pause survives the commit). Also adds TestSession.pauseAgent/resumeAgent.

bun run ts:build clean, full sdk suite 1642 pass / 0 fail.

---

Additional fixes (session-hardening round)

Five further commits, from analysing a real 5-week customer session (project skolkaeduart-sgp45j, 1073 LLM calls, $37.51). Independent of the CORR findings above.

preserve tool turns across context compaction — the root cause of 3 hard inference failures

Compaction could not represent tool structure: newConversationHistory was typed {role: 'user'|'assistant'|'system', content: string}, so createContextCompactedEvent dropped toolCalls and flattened tool → system.

conversationHistory is rewritten by compaction; pendingToolResults is not. A tool result that had already completed but not yet been consumed was appended after the compacted history (buildLLMMessages: [preamble, conversationHistory, pendingMessages]) with its declaring tool_use erased. OpenRouter discarded the unmatched tool message, the request then ended on an assistant message, and every route rejected it:

invalid_request_error: This model does not support assistant message prefill.
The conversation must end with a user message.

Verified signature over all 1073 calls in that session: exactly 4 requests matched "starts with [CONVERSATION SUMMARY] + no assistant carries toolCalls + last message is an unmatched role: 'tool'" — 3 of those 4 failed, and nothing else in the session failed.

CompactedConversationMessage is now = LLMMessage (zod discriminated union on role), old events still parse and reduce byte-identically, and computeKeepCount gained findOpenToolTurnStart so an assistant message with an unresolved tool turn is never cut out from under its result.

sanitize provider message histories — defence in depth

New message-sanitization.ts, shared by both providers: an orphaned tool result becomes a marked user message (data preserved, conversation ends on user) with a warning log, and a history still ending on assistant throws ProviderMessageValidationError → invalid_request rather than burning five provider routes on a guaranteed 400.

stop communication-only agent loops + make no-progress loop stops recoverable

A subagent that received an upload with no task sent its parent a reworded "still waiting for instructions" every ~9s — 39 messages, 41% of everything it ever sent, at an 87k-token average prompt. The existing guard only caught byte-identical repeats, which rewording evaded.

The first attempt regressed: replaying the rule over the real event log would have paused the orchestrator 18 times during legitimate work, because the counter reset on inbound mailbox messages but the orchestrator's inbound is user chat. Fixed by a generic seam — agent_input_consumed, emitted from consumeDequeuedTokens after every dequeue plugin's markConsumed, so user chat, mailbox, uploads and any future inbound source all reset it. Replay after the fix: 0 orchestrator trips, 12 subagent trips landing exactly on the real spin windows.

pause was also the wrong action — it has no in-band recovery (parents get <child-paused>, no resume tool; only the external agents.resume). Replaced with a soft stop: limit_warning + WAITING substitution, agent stays pending, new inbound work wakes it. Other hard limits keep their existing pause/human-recovery convention.

wake idle agents for ready uploads

A customer answered an ask_user question by uploading 8 photos instead of typing. The uploads plugin declared hasPendingMessages correctly, but nothing re-ran decide() — user chat wakes agents via an explicit scheduleAgent call, not generic event handling. The uploads sat unprocessed for 45 hours, until an unrelated agent restart consumed them and finished the work in two minutes.

Verification

bun run ts:build clean, bun run lint clean (one pre-existing unrelated warning), full suite 904 pass / 8 skip / 0 fail after rebase onto 6d21f5e2.