fix(responses): tolerate dict response payloads in streaming events

[devin-ai-integration]

TLDR

Problem this solves:

• Responses API streaming 500s on non-compliant provider payloads
• AttributeError: 'dict' object has no attribute 'usage' mid-stream
• Successful streamed requests silently missing from SpendLogs
• response.output: null crashes the /chat/completions bridge

How it solves it:

• Keep event.response typed when validation falls back to model_construct
• Read/write usage through helpers that tolerate raw dicts
• Treat a null output as an empty list

Relevant issues

Fixes #34754

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Repro setup: a local OpenAI-compatible upstream that streams a non-compliant response.completed (no created_at, output: null, usage as a plain dict), which is exactly the payload shape that makes litellm fall back to model_construct; the happy path is a real, paid gpt-5.6 call through the same proxy

proof_config.yaml

model_list:
  - model_name: bad-provider
    litellm_params:
      model: openai/bad-provider-model
      api_base: http://127.0.0.1:8123/v1
      api_key: sk-not-needed
  - model_name: real-openai
    litellm_params:
      model: openai/gpt-5.6
      api_key: os.environ/OPENAI_API_KEY

Before, at 2412326 (python litellm/proxy/proxy_cli.py --config proof_config.yaml --detailed_debug)

$ curl -sN -X POST http://127.0.0.1:4000/v1/responses -H 'content-type: application/json' \
    -H 'authorization: Bearer sk-1234' -d '{"model":"bad-provider","input":"hi","stream":true}'
data: {"type":"response.output_text.delta",...,"delta":"hi from the non-compliant provider","sequence_number":1,"model":"bad-provider"}

data: {"type":"response.completed","response":{"id":"resp_...","model":"bad-provider-model","object":"response","status":"completed","usage":{"input_tokens":11,"output_tokens":7,"total_tokens":18}},"sequence_number":2,"model":"bad-provider"}

data: [DONE]

proxy log; the event was built with model_construct, so the response stayed a raw dict and the spend entry lost its cost

transformation.py:364 - Pydantic validation failed for ResponseCompletedEvent with chunk {...}, falling back to model_construct
litellm_logging.py:1474 - response_cost_failure_debug_information: {'error_str': "'NoneType' object is not iterable", 'traceback_str': '...
  File "litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py", line 432, in response_includes_output_type
    for output_item in output:
TypeError: \'NoneType\' object is not iterable', 'model': 'bad-provider-model', 'call_type': 'aresponses'}
litellm_logging.py:2501 - Model=bad-provider-model; cost=None

After, at 4859517, same curl; the stream is unchanged and the log no longer carries a cost failure

litellm_logging.py:1458 - response_cost: 0.0
litellm_logging.py:2501 - Model=bad-provider-model; cost=0.0

$ grep -c response_cost_failure_debug_information proxy_after.log
0

Real provider sanity check on the same build, a paid streaming call to OpenAI

$ curl -sN -X POST http://127.0.0.1:4000/v1/responses -H 'content-type: application/json' \
    -H 'authorization: Bearer sk-1234' -d '{"model":"real-openai","input":"say hi in 3 words","stream":true}'
data: {"type":"response.completed","response":{"id":"resp_...","created_at":1785140645,"model":"gpt-5.6-sol","object":"response","output":[...],"status":"completed","usage":{...}},...}

# proxy log
litellm_logging.py:2501 - Model=gpt-5.6; cost=0.00123

The two dict-usage paths (Router._extract_partial_responses_usage and Logging._get_assembled_streaming_response) need a provider that both fails validation and drops mid-stream, which I could not force against a live provider; they are covered by the regression tests instead

Type

🐛 Bug Fix

Changes

The root cause is in OpenAIResponsesAPIConfig.transform_streaming_response: when a provider's response.completed payload fails pydantic validation, the code falls back to event_pydantic_model.model_construct(**parsed_chunk), which skips validation for nested fields too. The event then carries a plain dict where a ResponsesAPIResponse is declared, and every downstream event.response.usage access blows up.

except ValidationError:
    return event_pydantic_model.model_construct(**_with_typed_response_payload(parsed_chunk))

def _with_typed_response_payload(parsed_chunk):
    response_payload = parsed_chunk.get("response")
    if not isinstance(response_payload, dict):
        return parsed_chunk
    try:
        response = ResponsesAPIResponse(**response_payload)
    except ValidationError:
        response = ResponsesAPIResponse.model_construct(**response_payload)
        # nested usage is unvalidated too, so coerce it as well
        ...
    return {**parsed_chunk, "response": response}

Cost tracking had the same null-output crash (StandardBuiltInToolCostTracking.response_includes_output_type iterating response_object.output), which silently zeroed out the spend entry, so it takes the same or [] treatment.

The consumers are hardened as well, since nothing stops another code path from handing them an unvalidated payload. litellm/responses/utils.py grows normalize_response_api_usage, get_response_api_usage and set_response_api_usage; the router's _extract_partial_responses_usage / _combine_responses_fallback_usage and Logging._get_assembled_streaming_response go through those instead of reaching for .usage directly. The logging path previously only transformed usage when it was already a ResponseAPIUsage instance, so a dict-shaped usage silently produced no chat-style usage; it now normalizes first, which is what keeps the spend log entry.

Last, the /responses -> /chat/completions bridge treated response.output as always iterable. response_data.get("output", []) returns None when the key is present and null, so for item in output_items raised. It now falls back to an empty list, and the completed chunk still reports usage and finish_reason.

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/b310da521394445bb32dfb071ff7fdd5

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR hardens Responses API streaming against unvalidated dictionary payloads and null output values.

• Preserves typed nested response objects when stream-event validation falls back to model_construct.
• Centralizes normalization, reading, and mutation of Responses API usage data.
• Uses normalized usage in router fallback accounting and assembled streaming logs.
• Treats a null completed-response output as an empty list in the chat-completions bridge.
• Adds regression coverage for typed response recovery, dictionary usage payloads, fallback accounting, logging, and null output.

Confidence Score: 4/5

The usage normalization defect should be fixed before merging because malformed nested details currently remove valid token totals from fallback and logging accounting.

The helper validates the complete usage object atomically and returns no usage when a nested detail fails validation, even when all top-level token counts are valid, so downstream aggregation and spend logging undercount the request.

Files Needing Attention: litellm/responses/utils.py, litellm/router.py, litellm/litellm_core_utils/litellm_logging.py

Important Files Changed

Filename	Overview
litellm/llms/openai/responses/transformation.py	Reconstructs the nested response and usage models before returning an otherwise unvalidated streaming event.
litellm/responses/utils.py	Adds shared usage helpers, but whole-object validation discards valid top-level token totals when nested usage details are malformed.
litellm/router.py	Routes streaming fallback usage extraction and combination through the new helpers, inheriting their all-or-nothing normalization behavior.
litellm/litellm_core_utils/litellm_logging.py	Normalizes dictionary response and usage payloads while assembling completed streams, but malformed nested usage details still remove valid totals.
litellm/completion_extras/litellm_responses_transformation/transformation.py	Safely interprets a null completed-response output as empty when determining the bridged finish reason.

_{Reviews (1): Last reviewed commit: "fix(responses): tolerate dict response p..." | Re-trigger Greptile}

[greptile-apps]

Malformed details discard token totals

When a dict contains valid input_tokens, output_tokens, and total_tokens but a malformed nested details field, validating the entire object raises and this helper returns None, causing fallback aggregation and streaming logging to discard the valid top-level counts and underreport usage and spend.

Knowledge Base Used:

• Logging & Observability Integrations
• Router: deployment selection, retries, fallbacks, and cooldowns

[codecov]

Codecov Report

❌ Patch coverage is 91.93548% with 5 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/responses/utils.py	90.00%	3 Missing ⚠️
litellm/router.py	71.42%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_responses_streaming_dict_payload (4859517) with litellm_internal_staging (2412326)}