Skip to content

feat(openrouter): native content-block streaming events#38030

Open
Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
nh/native-content-block-streamingfrom
nh/native-content-block-streaming-openrouter
Open

feat(openrouter): native content-block streaming events#38030
Nick Hollon (nick-hollon-lc) wants to merge 1 commit into
nh/native-content-block-streamingfrom
nh/native-content-block-streaming-openrouter

Conversation

@nick-hollon-lc

Copy link
Copy Markdown
Contributor

Stacked on #37995 (core BlockStreamTracker + Anthropic), which it branches from. Depends only on the core primitive — sibling of the OpenAI chain. Retargets to master once #37995 lands.

Gives ChatOpenRouter native _stream_chat_model_events / _astream_chat_model_events hooks so stream_events(version="v3") maps OpenRouter's stream directly to content-block events.

How it works

A bespoke converter (convert_openrouter_stream / async twin), sibling of the groq converter, builds text, reasoning, and tool-call blocks directly from OpenRouter's raw OpenAI-shaped delta, feeding the shared BlockStreamTracker. Tool-call args stream incrementally and finalize to a parsed dict. It does not depend on langchain-openai — it reuses only OpenRouter's own _create_usage_metadata.

OpenRouter specifics it handles: error chunks are propagated as a ValueError (matching _stream); top-level usage is accumulated; and OpenRouter's cost surfacing (cost/cost_details) plus native_finish_reason/model_name/etc. are carried into message-finish metadata, matching the compat bridge.

The win over the bridge

OpenRouter's chunk parser leaves reasoning in additional_kwargs; the native converter surfaces it as a reasoning block (and tool calls as tool_call blocks).

Worth careful review

  • Error-chunk propagation and the cost/finish metadata parity with _stream (a test asserts cost survives to message-finish).
  • The _StreamState helper shares per-chunk coercion/usage/error logic across the sync/async twins (no event-sequence drift).
  • Scope cut: reasoning_details not yet surfaced as blocks (noted in-code).
  • message_id keyword-only; message-start carries the LangChain run id.

@github-actions github-actions Bot added feature For PRs that implement a new feature; NOT A FEATURE REQUEST integration PR made that is related to a provider partner package integration internal openrouter `langchain-openrouter` package issues & PRs size: L 500-999 LOC labels Jun 10, 2026

@open-swe open-swe Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Open SWE Review found 1 potential issue.

Open in WebView Open SWE trace

Comment on lines +170 to +171
if choice.get("finish_reason"):
state.record_finish(chunk_dict, choice)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Preserve streamed logprobs metadata

When the caller requests streaming logprobs, OpenRouter includes them on choice["logprobs"]. The existing _stream path copies that into generation_info, and the v3 compat bridge merges it into message-finish.metadata; after this native hook is added, stream_events(version="v3", logprobs=True, ...) takes this converter instead and line 170 only records finish metadata, so those logprobs are dropped from the event stream entirely. Please accumulate/propagate choice.get("logprobs") here (and in the async twin) to keep parity with the bridge.

(Refers to lines 170-171)


Your feedback helps Open SWE learn. React with 👍 or 👎 to tell us if this review comment was useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature For PRs that implement a new feature; NOT A FEATURE REQUEST integration PR made that is related to a provider partner package integration internal openrouter `langchain-openrouter` package issues & PRs size: L 500-999 LOC

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant