feat(groq): native content-block streaming events#38029
feat(groq): native content-block streaming events#38029Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
Conversation
| if finish_reason := choice.get("finish_reason"): | ||
| response_metadata["finish_reason"] = finish_reason |
There was a problem hiding this comment.
🟡 Logprobs dropped from v3 streams
The native converter only carries finish_reason into the final message-finish.metadata. Before this hook was added, Groq v3 events used the compat bridge over _stream(), and _stream() preserves choice["logprobs"] in generation_info, which chunks_to_events() forwards into the final metadata. With the new path, ChatGroq(...).stream_events(..., version="v3", logprobs=True) silently loses the returned logprobs even though they are present on the raw choice. The async twin has the same issue at lines 144-145, so both converters should copy choice.get("logprobs") into response_metadata like _stream() does.
(Refers to lines 109-110)
Your feedback helps Open SWE learn. React with 👍 or 👎 to tell us if this review comment was useful.
a3ef4d9 to
f7d7e0b
Compare
b55849a to
01d9cf7
Compare
Gives
ChatGroqnative_stream_chat_model_events/_astream_chat_model_eventshooks sostream_events(version="v3")maps groq's stream directly to content-block events.How it works
A bespoke converter (
convert_groq_stream/ async twin) builds text, reasoning, and tool-call blocks directly from groq's raw OpenAI-shaped delta, feeding the sharedBlockStreamTracker. groq streams tool-call arguments incrementally (id+name on the first delta, fragments on later deltas at the sameindex), so the tracker accumulates them and finalizes to a parsed dict. Usage is read fromx_groq.usage; thearguments == "null"quirk is normalized to"{}".It deliberately does not depend on
langchain-openai— it reuses only groq's own_create_usage_metadata.The win over the bridge
groq's existing chunk parser puts
tool_callsandreasoninginadditional_kwargs, so the compat bridge surfaces neither as content blocks. The native converter surfaces both — tool calls astool_callblocks and reasoning asreasoningblocks.Worth careful review
tool:{index}); a test covers parallel tool calls at distinct indices.message_idis keyword-only on the hooks, kept out of the request kwargs;message-startcarries core's LangChain run id.executed_tools) are not yet surfaced as blocks (noted in-code); the bridge still carried them inadditional_kwargs.