Skip to content

feat(groq): native content-block streaming events#38029

Open
Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
nh/native-content-block-streamingfrom
nh/native-content-block-streaming-groq
Open

feat(groq): native content-block streaming events#38029
Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
nh/native-content-block-streamingfrom
nh/native-content-block-streaming-groq

Conversation

@nick-hollon-lc

Copy link
Copy Markdown
Contributor

Stacked on #37995 (core BlockStreamTracker + Anthropic), which it branches from. Depends only on the core primitive — sibling of the OpenAI chain. Retargets to master once #37995 lands.

Gives ChatGroq native _stream_chat_model_events / _astream_chat_model_events hooks so stream_events(version="v3") maps groq's stream directly to content-block events.

How it works

A bespoke converter (convert_groq_stream / async twin) builds text, reasoning, and tool-call blocks directly from groq's raw OpenAI-shaped delta, feeding the shared BlockStreamTracker. groq streams tool-call arguments incrementally (id+name on the first delta, fragments on later deltas at the same index), so the tracker accumulates them and finalizes to a parsed dict. Usage is read from x_groq.usage; the arguments == "null" quirk is normalized to "{}".

It deliberately does not depend on langchain-openai — it reuses only groq's own _create_usage_metadata.

The win over the bridge

groq's existing chunk parser puts tool_calls and reasoning in additional_kwargs, so the compat bridge surfaces neither as content blocks. The native converter surfaces both — tool calls as tool_call blocks and reasoning as reasoning blocks.

Worth careful review

  • Tool-arg accumulation across deltas (keyed by tool:{index}); a test covers parallel tool calls at distinct indices.
  • message_id is keyword-only on the hooks, kept out of the request kwargs; message-start carries core's LangChain run id.
  • Scope cut: groq server-tool results (executed_tools) are not yet surfaced as blocks (noted in-code); the bridge still carried them in additional_kwargs.
  • Sync/async are faithful twins.

@github-actions github-actions Bot added feature For PRs that implement a new feature; NOT A FEATURE REQUEST groq `langchain-groq` package issues & PRs integration PR made that is related to a provider partner package integration internal size: L 500-999 LOC labels Jun 10, 2026

@open-swe open-swe Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open SWE Review found 1 potential issue.

Open in WebView Open SWE trace

Comment on lines +109 to +110
if finish_reason := choice.get("finish_reason"):
response_metadata["finish_reason"] = finish_reason

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Logprobs dropped from v3 streams

The native converter only carries finish_reason into the final message-finish.metadata. Before this hook was added, Groq v3 events used the compat bridge over _stream(), and _stream() preserves choice["logprobs"] in generation_info, which chunks_to_events() forwards into the final metadata. With the new path, ChatGroq(...).stream_events(..., version="v3", logprobs=True) silently loses the returned logprobs even though they are present on the raw choice. The async twin has the same issue at lines 144-145, so both converters should copy choice.get("logprobs") into response_metadata like _stream() does.

(Refers to lines 109-110)


Your feedback helps Open SWE learn. React with 👍 or 👎 to tell us if this review comment was useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature For PRs that implement a new feature; NOT A FEATURE REQUEST groq `langchain-groq` package issues & PRs integration PR made that is related to a provider partner package integration internal size: L 500-999 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant