Skip to content

feat(groq): native content-block streaming events#8

Open
LennyMalcolm0 wants to merge 3 commits into
masterfrom
devasign-clone/pr-38029
Open

feat(groq): native content-block streaming events#8
LennyMalcolm0 wants to merge 3 commits into
masterfrom
devasign-clone/pr-38029

Conversation

@LennyMalcolm0

@LennyMalcolm0 LennyMalcolm0 commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Stacked on #14 (core BlockStreamTracker + Anthropic), which it branches from. Depends only on the core primitive — sibling of the OpenAI chain. Retargets to master once #14 lands.

Gives ChatGroq native _stream_chat_model_events / _astream_chat_model_events hooks so stream_events(version="v3") maps groq's stream directly to content-block events.

How it works

A bespoke converter (convert_groq_stream / async twin) builds text, reasoning, and tool-call blocks directly from groq's raw OpenAI-shaped delta, feeding the shared BlockStreamTracker. groq streams tool-call arguments incrementally (id+name on the first delta, fragments on later deltas at the same index), so the tracker accumulates them and finalizes to a parsed dict. Usage is read from x_groq.usage; the arguments == "null" quirk is normalized to "{}".

It deliberately does not depend on langchain-openai — it reuses only groq's own _create_usage_metadata.

The win over the bridge

groq's existing chunk parser puts tool_calls and reasoning in additional_kwargs, so the compat bridge surfaces neither as content blocks. The native converter surfaces both — tool calls as tool_call blocks and reasoning as reasoning blocks.

Worth careful review

  • Tool-arg accumulation across deltas (keyed by tool:{index}); a test covers parallel tool calls at distinct indices.
  • message_id is keyword-only on the hooks, kept out of the request kwargs; message-start carries core's LangChain run id.
  • Scope cut: groq server-tool results (executed_tools) are not yet surfaced as blocks (noted in-code); the bridge still carried them in additional_kwargs.
  • Sync/async are faithful twins.

Cloned from langchain-ai#38029 for DevAsign stress testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants