fix(analyze): a budget-exhausted empty completion retries once at a raised, SDK-safe cap (#569, choice c)

[gadievron]

Summary

The #292 decision classified every empty-completion error as same-cap-retryable — a rationale that predates #561's cause split. Since #561 the message for a length-stop empty states the deterministic cause ("the output budget was consumed before any visible content"), and the classifier retried it at the same cap anyway: a billed coin flip against a cause the message itself names.

The fix (the choice-(c) ruling — both review seats converged):

• is_budget_exhausted_error discriminates the class by the budget wording across all adapters (the openai chat + anthropic "output budget was consumed"; gemini-thinking "consumed the token budget"; the o-series Responses path "reasoning consumed the budget") — nothing is declassified; only the retry's cap differs;
• the analyze retry loop splits: the budget class retries once at BUDGET_RETRY_MAX_TOKENS = min(2× default, 21000) — raised above the default but under the Anthropic non-streaming ceiling (~21,333; the SDK rejects higher with "Streaming is required", which would make the retry a guaranteed failure on that adapter — the adversarial review's blocker, folded); every other retryable keeps the is_retryable_error honours "no usable content" only on the string path, so enhance never retries the one error class that actually occurs #292 same-cap re-roll. The retry log line names the cap used; the per-index decision is a testable production helper;
• the cap threads through _process_unit → analysis_core.analyze_unit → simple_text (None = the unchanged default, byte-identical);
• the contradictory stale CONC-C2 notes unified (post-A rejected completion's returned usage is discarded before it is read; a verify conversation's accumulated usage is lost on exceptional exit #537/The anthropic adapter's empty-completion raise carries no usage — the #537 usage carriage exists on the openai sites only #564 the rejected reply's usage rides the error — a retry is billed and recorded).

Named deferred: context_enhancer.py's retry loop is the same class (its single-shot call pins 4096) — a follow-up; a true 2× cap needs a provider-layer streaming/timeout change (the SDK ceiling).

Closes #569.

Test plan

12 tests: the discriminator (the length/stop/transient/dict shapes + the three adapters' parity wordings + the filtered class not budget); the cap chain exists; the SDK-ceiling pin (above the default, at/below 21000); the production budget_retry_cap decision; the end-to-end split (a length-empty unit's retry carries the raised cap, a stop-empty unit's carries None).

Verification evidence

check	command	result
new tests	pytest tests/test_issue569_budget_retry.py -q	12 passed
full suite	pytest tests/ -q	4029 passed, 2 failed — both pre-existing (SDK pin drift, verified on the pristine base)
static analysis	ruff	clean
adversarial review	combined wave+refute	BLOCK → the 5 findings folded: the Anthropic SDK ceiling blocker (the cap clamped to 21000, the constant renamed to say so), the dead test (the production helper extracted + the real drive), the parity gap (gemini/Responses markers), the per-iteration set, the contradictory stale note

[gadievron]

Verification review — recommend-merge. (all evidence pinned to head 570ddf9; re-verify on any new head)

The fix: is_budget_exhausted_error (rate_limiter.py) discriminates the deterministic budget-exhaustion empty completions by the adapter wording; run_analysis's retry pass gives those units ONE retry at BUDGET_RETRY_MAX_TOKENS = min(2×DEFAULT, 21000) (under the Anthropic non-streaming ceiling), threaded through _process_unit → analyze_unit → simple_text. Every other retryable keeps the #292 same-cap rationale.

The review round, fixed on this branch (570ddf9):

• The gemini and OpenAI-Responses empty-completion raises emitted the budget wording UNCONDITIONALLY, so the filtered/malformed class also took the raised cap on two of four adapters. Both producers now branch on the finish signal (google: raw_finish in the MAX_TOKENS forms; Responses: stop_reason == "max_tokens"), mirroring the openai-chat #561 pattern — the truncated wordings carry the marker, the filtered wordings do not. Producer-side pins added in both adapter suites (driving the real raises, positive and negative).
• The "end-to-end" retry-loop test re-implemented the loop and never invoked run_analysis — deleting the wiring left every test green. It now drives the REAL run_analysis retry pass (stubbed _run_detection + a _process_unit capture) and asserts the raised cap threads only into the budget-class unit. Must-trip verified: with the wiring line deleted the test fails; restored it passes.

Verification (run at the pinned head): targeted slices (issue569 + google/openai adapters + retry-empty-completion) — 107 passed; full suite — 4037 passed, 33 skipped, 0 FAIL; CI at 570ddf9: tests + gitleaks success. Cross-PR: the combined tree of all six open PRs passed the full suite (3912, pre-delta); no retry stacking with #575 (the raised cap is analyze-phase-only; LLR failures go to batches_failed → absence-as-retry).

Known trade-offs (documented, accepted by the review round): the raised cap is +5% over the 20000 default — efficacy-limited against deep reasoning exhaustion (an Anthropic-envelope constraint; a per-adapter ceiling would raise it for the other providers); the min() clamps non-Anthropic providers to Anthropic's ceiling.

Follow-ups (named, none blocking):

• The Responses empty-content marker gate could key on the raw incomplete_details.reason == "max_output_tokens" instead of the derived stop_reason — the unknown-incomplete/unknown-status relabels also set max_tokens, so those exotic shapes carry the budget wording as a false causal statement (retry direction safe; message honesty only).
• The same deterministic budget class remains same-cap in verify/enhance and LLR (scoped out; the cheap LLR composition is to split the batch when _attempt's exception arm sees is_budget_exhausted_error).
• The resume path re-runs budget-error checkpoint rows at the default cap before the retry pass (pre-existing shape).
• test_retry_empty_completion.py's _EMPTY_GEMINI literal is stale vs the current producer wordings (non-load-bearing — retryability rides on the shared substrings); _EMPTY_OPENAI_RESPONSES_FILTERED is defined but not asserted.

Approve + merge are code-owner actions; never on red CI.