feat(openai): native content-block streaming for chat completions

[Nick Hollon (nick-hollon-lc)]

Stacked on #37995 (the BlockStreamTracker foundation + Anthropic reference). Review/merge that first; this targets its branch and will retarget to master once it lands.

Builds on the shared BlockStreamTracker to give ChatOpenAI native content-block streaming events for the Chat Completions path, so stream_events(version="v3") no longer rides the compat bridge for plain OpenAI streaming.

What it does

A thin pure converter (convert_openai_completions_stream / async twin) drives raw OpenAI Chat Completions chunks into the shared tracker, reusing the existing _convert_chunk_to_generation_chunk for content extraction — the same two-layer pattern Anthropic uses. BaseChatOpenAI gains _stream_chat_model_events / _astream_chat_model_events (the hooks core detects); ChatOpenAI and Azure inherit them. The converters are exported so the OpenAI-compatible providers (deepseek, fireworks, groq, xai, openrouter) can reuse them in follow-up PRs — either as a direct pass-through or via a small chunk adapter.

Scope and deferrals

This PR specializes the Chat Completions path only. The hooks defer to the existing compat bridge — unchanged behavior — when any of these apply:

• the Responses API is active (_use_responses_api); a native Responses converter is the next PR,
• structured output (response_format, including when set via model_kwargs), whose get_final_completion() flow the native path doesn't reproduce,
• raw-header mode (include_response_headers).

The deferral guard mirrors ChatOpenAI._stream's own dispatch so the Responses path is never hijacked.

Worth careful review

• The deferral guard in both hooks — it must match _stream's routing exactly. A test (test_response_format_in_model_kwargs_takes_bridge_path) is mutation-verified to fail if the model_kwargs branch of the guard is removed.
• The async hook preserves stream_chunk_timeout by wrapping the stream the same way _astream does.
• Parity is anchored by the pre-existing test_openai_stream_events_v3_lifecycle, which previously exercised the bridge and now routes through the native hook unchanged (plus a new async twin).

Known limitation

On the v3 events path on_llm_new_token is called with text only (the legacy chunk= / logprobs= kwargs aren't threaded through the pure converter); the lifecycle events themselves carry the block and usage detail. Flagged in-code.

[open-swe]

Open SWE Review found 2 potential issues.

Open in Web • View Open SWE trace