.NET: Add RoutingChatClient for routing requests across multiple chat clients#6932
.NET: Add RoutingChatClient for routing requests across multiple chat clients#6932westey-m wants to merge 4 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR adds a new experimental .NET IChatClient decorator (RoutingChatClient) that can route each request to one of multiple named inner chat clients based on per-session routing state, enabling model/provider switching mid-conversation without losing client-side chat history.
Changes:
- Introduces
RoutingChatClientplus supporting types (RoutingChatClientOptions,RoutingContext,RoutingState) and registers routing state for JSON serialization. - Adds a full unit test suite for routing behavior, session isolation, and fallback client lifecycle/disposal.
- Adds a new sample (
Agent_Step22_MultiModelRouting) and wires it into the samples README and solution.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| dotnet/src/Microsoft.Agents.AI/ChatClient/RoutingChatClient.cs | Implements the routing decorator, per-session state, fallback factory behavior, and disposal semantics. |
| dotnet/src/Microsoft.Agents.AI/ChatClient/RoutingChatClientOptions.cs | Adds configuration knobs for routing callback, state key, and fallback disposal behavior. |
| dotnet/src/Microsoft.Agents.AI/ChatClient/RoutingContext.cs | Defines the callback context passed to router/fallback factory. |
| dotnet/src/Microsoft.Agents.AI/ChatClient/RoutingState.cs | Defines the session-serializable routing state. |
| dotnet/src/Microsoft.Agents.AI/AgentJsonUtilities.cs | Registers RoutingState for source-generated JSON serialization. |
| dotnet/tests/Microsoft.Agents.AI.UnitTests/ChatClient/RoutingChatClientTests.cs | Adds unit tests for routing, session scoping, fallback behavior, and disposal. |
| dotnet/samples/02-agents/Agents/README.md | Adds the Step 22 sample to the samples list. |
| dotnet/samples/02-agents/Agents/Agent_Step22_MultiModelRouting/README.md | Documents the new multi-model routing sample and key concepts. |
| dotnet/samples/02-agents/Agents/Agent_Step22_MultiModelRouting/Program.cs | Implements the sample showing switching active model and using fallback factory. |
| dotnet/samples/02-agents/Agents/Agent_Step22_MultiModelRouting/Agent_Step22_MultiModelRouting.csproj | Adds the sample project and references. |
| dotnet/agent-framework-dotnet.slnx | Includes the new sample project in the solution. |
| using Microsoft.Shared.DiagnosticIds; | ||
| using Microsoft.Shared.Diagnostics; | ||
|
|
||
| namespace Microsoft.Agents.AI; |
There was a problem hiding this comment.
Suggest proposing this into Microsoft.Extensions.AI package as the source of this abstraction as a MEAI001 experiment seems to be a better fit.
Additionally if this implementation is intended to be rapidly used by the Harness, I would keep it in a separate folder for MEAI-ChatClient's like we had before when proposing different IChatClient impl.
| namespace Microsoft.Agents.AI; | |
| namespace Microsoft.Extensions.AI; |
There was a problem hiding this comment.
At first a ChatClient should not be aware of agents except if used internally, we do that in the OTEL Agent to reuse some logic. I think proposing this as a public API opens a wrong precedent how IChatClient should be implemented.
Motivation & Context
Applications that let a user switch the underlying model/provider mid-conversation (e.g. from OpenAI to Anthropic, or between two models) lose the conversation context on the new provider, forcing manual rehydration of the chat history. This is exactly the scenario described in #6865.
This PR introduces
RoutingChatClient, an experimentalIChatClientdecorator (.NET) that holds multiple named inner chat clients and routes each request to one of them based on a per-session active destination. Because the conversation is carried client-side by the agent's session, switching destinations mid-conversation preserves the full history for whichever client handles the next turn — no manual rehydration needed.Description & Review Guide
RoutingChatClient(Microsoft.Agents.AI/ChatClient/): anIChatClientdecorator over multiple named inner clients, with:GetActiveDestinationKey/SetActiveDestinationKey(null key falls back to the first inner client, then to the fallback factory).ValueTask)Routercallback for custom heuristics (e.g. inference-based selection).DisableFallbackChatClientDisposal).RoutingChatClientOptions,RoutingContext,RoutingState, plus JSON serialization registration inAgentJsonUtilities.RoutingChatClientTests).Agent_Step22_MultiModelRoutingdemonstrating multi-model routing, client-side chat history, disabling encrypted reasoning content across models, and retrieving the routing client viaGetService(added to the solution and the Agents README).MAAI001). No existing behavior is changed.InvalidOperationException).Related Issue
References #6865. This PR implements the .NET side of the request; the Python side remains tracked by the same issue, so it is intentionally referenced without a closing keyword.
Contribution Checklist
breaking changelabel (or add "[BREAKING]" to the title prefix, before or after any language prefix) — a workflow keeps the label and title prefix in sync automatically.