Skip to content

fix: handle dict response in Responses API streaming and logging#34759

Open
Ericcwang23 wants to merge 2 commits into
BerriAI:mainfrom
Ericcwang23:fix-responses-v2
Open

fix: handle dict response in Responses API streaming and logging#34759
Ericcwang23 wants to merge 2 commits into
BerriAI:mainfrom
Ericcwang23:fix-responses-v2

Conversation

@Ericcwang23

Copy link
Copy Markdown

Fixes #34754

Three code paths assumed response is a Pydantic object but some upstream providers return a plain dict, causing AttributeError: 'dict' object has no attribute 'usage'.

  • router.py: add isinstance(completed.response, dict) guard before .usage access
  • litellm_logging.py: add isinstance(result.response, dict) guard before .usage access
  • transformation.py: add or [] fallback for None output_items

@greptile-apps

greptile-apps Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds defensive handling for dictionary-backed Responses API streaming events and nullable output collections.

  • Extracts usage from dictionary responses in the Router fallback path.
  • Allows logging to assemble dictionary-backed completed responses.
  • Treats absent response output collections as empty while transforming responses.

Confidence Score: 4/5

The PR appears safe to merge, with only a non-blocking return-type annotation mismatch in the logging helper.

The changed runtime paths safely avoid attribute access on dictionary responses and handle absent output collections; the remaining issue is that _get_assembled_streaming_response now returns a dictionary excluded by its annotation.

Files Needing Attention: litellm/litellm_core_utils/litellm_logging.py

Important Files Changed

Filename Overview
litellm/completion_extras/litellm_responses_transformation/transformation.py Safely skips nullable output collections in non-streaming and terminal streaming transformations without changing established empty-output behavior.
litellm/litellm_core_utils/litellm_logging.py Adds runtime support for dictionary-backed completed responses, but the helper's return annotation no longer describes every returned type.
litellm/router.py Defensively extracts nested usage from dictionary responses while preserving the existing typed response path.

Reviews (1): Last reviewed commit: "fix: handle dict response in Responses A..." | Re-trigger Greptile

):
## return unified Usage object
if isinstance(result.response, dict):
return result.response

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Return annotation excludes dictionaries

This branch returns a plain dictionary even though _get_assembled_streaming_response declares only model response types or None, making the type contract inaccurate and preventing callers from correctly inferring dictionary handling requirements.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 33.33333% with 4 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
litellm/litellm_core_utils/litellm_logging.py 0.00% 2 Missing ⚠️
litellm/router.py 0.00% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@codspeed-hq

codspeed-hq Bot commented Jul 27, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing Ericcwang23:fix-responses-v2 (97aa248) with main (0cd588a)

Open in CodSpeed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant