fix: handle dict response in Responses API streaming and logging#34759
fix: handle dict response in Responses API streaming and logging#34759Ericcwang23 wants to merge 2 commits into
Conversation
Greptile SummaryThis PR adds defensive handling for dictionary-backed Responses API streaming events and nullable output collections.
Confidence Score: 4/5The PR appears safe to merge, with only a non-blocking return-type annotation mismatch in the logging helper. The changed runtime paths safely avoid attribute access on dictionary responses and handle absent output collections; the remaining issue is that Files Needing Attention: litellm/litellm_core_utils/litellm_logging.py
|
| Filename | Overview |
|---|---|
| litellm/completion_extras/litellm_responses_transformation/transformation.py | Safely skips nullable output collections in non-streaming and terminal streaming transformations without changing established empty-output behavior. |
| litellm/litellm_core_utils/litellm_logging.py | Adds runtime support for dictionary-backed completed responses, but the helper's return annotation no longer describes every returned type. |
| litellm/router.py | Defensively extracts nested usage from dictionary responses while preserving the existing typed response path. |
Reviews (1): Last reviewed commit: "fix: handle dict response in Responses A..." | Re-trigger Greptile
| ): | ||
| ## return unified Usage object | ||
| if isinstance(result.response, dict): | ||
| return result.response |
There was a problem hiding this comment.
Return annotation excludes dictionaries
This branch returns a plain dictionary even though _get_assembled_streaming_response declares only model response types or None, making the type contract inaccurate and preventing callers from correctly inferring dictionary handling requirements.
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
Fixes #34754
Three code paths assumed
responseis a Pydantic object but some upstream providers return a plaindict, causingAttributeError: 'dict' object has no attribute 'usage'.router.py: addisinstance(completed.response, dict)guard before.usageaccesslitellm_logging.py: addisinstance(result.response, dict)guard before.usageaccesstransformation.py: addor []fallback for Noneoutput_items