From 3feecf9df60df1f16f64d1709a25823d5aa58f07 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Sat, 7 Jun 2025 12:32:02 +1200 Subject: [PATCH 01/30] Routing API Signed-off-by: Tom Bentley --- proposals/004-routing-api.md | 248 +++++++++++++++++++++++++++++++++++ 1 file changed, 248 insertions(+) create mode 100644 proposals/004-routing-api.md diff --git a/proposals/004-routing-api.md b/proposals/004-routing-api.md new file mode 100644 index 00000000..9b131c75 --- /dev/null +++ b/proposals/004-routing-api.md @@ -0,0 +1,248 @@ + + +# A Routing API + +This proposal discusses an API for routing requests to Kafka clusters. + +## Current situation + +Kroxylicious currently supports `Filter` plugins as the top-level mechanism for adding behaviour to a proxy. +`Filters` can manipulate Kafka protocol requests and responses being sent by/to the Kafka client. +They can also originate requests of their own (for example to obtain metadata that's necessary for them to function). + +However, `Filters` cannot influence which Kafka cluster will receive the requests which it forwards or makes for itself. + +## Motivation + +There are use cases for a Kafka proxy which cannot be served with the `Filter` API alone. + +Here are some examples: + +* **Union clusters**. +Multiple backing Kafka clusters can be presented to clients as a single cluster. +Broker-side entities, such as topics, get bijectively mapped (for example using a per-backing cluster prefix) to the +virtual entites presented to clients. +`Filters` cannot easily do this because they're always hooked up to a single backing Kafka cluster. + +* **Topic splicing**. +Multiple separate topics in distinct backing clusters are presented to clients as a single topic. +Only one backing topic is writable at any given logical time. + +* **Principal-aware routing**. +A natural variation on basic SASL termination is to use the identity of the authenticated client to drive the decision about which backing cluter to route requests to. + +Kroxylicious is currently unable to address use cases like these. + +## Proposal + +### Concepts + +To enable the use cases above we need a few new concepts: + +* A _receiver_ is something that can handle requests, and which will return at most one response. +* A _route_ represents a possible pathway from an incoming request towards a receiver. +* A _router_ is a thing which decides which route should be used for a given request. + +Routers and backing Kafka clusters are both examples of receivers. +Slightly more generally, a receiver is anything that speaks the full Kafka procotol. +We don't consider a `Filter` to be a receiver, even though it can make short-circuit responses. +`Filters` usually rely on having a backing Kafka cluster to forward requests to. +Generally speaking, a `Filter` might only handle a subset of the `ApiKeys` of the Kafka protocol. +Routers and backing clusters necessarily handle the whole protocol. + +### Plugin API + +`Router` will be a top level plugin analogous to the `Filter` plugin interface, using the same `ServiceLoader`-based mechanism for runtime discovery. +Each `Router` implementation will support 0 or more named routes. +The available and required route names will depend on the implementation, which might ascribe different behaviour to different named routes. +For example a `Router` implementing the 'union cluster' use case might simply use the route names as prefixes for names of the broker-side entities +it will expose (such as topics or consumer groups), as as such impose no restriction on the supported route names. +In contrast, a `Router` implementing the 'topic splicing' use case might require configuration about each of the clusters being used in the splice, which +would required the route names to be referenced in the `Router`'s configuration. + +```java +interface Router { + CompletionStage onClientRequest(short apiVersion, + ApiKeys apiKey, + RequestHeaderData header, + ApiMessage request, + RoutingContext context); +} +``` + +For a given incoming request a `Router` implementation can decide which route(s) to make a request to. +We want to allow a router to potentially make multiple requests (e.g. to multiple clusters) and to have control over their processing (e.g. sequential or concurrent). +For this reason the `RoutingContext` does not follow the builder pattern used in the `FilterContext`, but simply +exposes methods to asynchronously send requests down a given route. +This allows the `Router` author to make use of the `CompletionStage` API when issuing multiple requests. + +```java +interface RoutingContext { + + CompletionStage sendRequest(String route, ...) + void sendResponse(Resonse) + void disconnect() + +} +``` + + +### Configuration + +Routers are configured at the top level of the proxy configuration, similarly to `filterDefinitions`: +In addition to the `name`, `type` and `config` (which serve the same purpose for `Routers` as they do for `Filters`), a `routerDefinition` also supports a `routes` property. +The `routes` property is optional, though any given implementation may have its own particular requirements for its `routes`. + +```yaml +routerDefinitions: + - name: my-router + type: MyRouter + config: ... + routes: + - name: foo + filters: + - my-first-filter + - my-second-filter + cluster: my-backing-cluster + - name: bar + filters: + - my-third-filter + router: my-other-router + - name: my-other-router + # ... +``` + +A route object has a `name`, an optional list of `filters` (being the names of the filters to be applied to requests/responses that traverse this route) and either a +`cluster` or a `router` property, which names the receiver which will handle requests after any filters have been applied. +Exactly one of `cluster` or `router` must be specified. + +Because routers can refer to other routers they form a graph. + +``` + my-backing-cluster + ^ + / + / foo + requests / + ---------> my-router + \ + \ bar + \ ... + v / + my-other-router --- ... + \ + ... +``` + +All _possible_ routes through the graph can be determined statically from the proxy configuration, but the routing of any individual incoming request is determined at runtime +can might involve multiple outgoing requests from any given router. +Validation performed at proxy startup will reject cyclic graphs. +This will prevent the possibility of a request getting stuck in a router loop. + +In order for non-trivial router graphs to be useful, `Router` authors will need to follow the same principle of composition as `Filter` authors. +That is, a `Router` implementation should only talk to its receivers, and not, for example, make direct connections of its own to a backing cluster. +To do so would prevent use of that router implementation in a larger graph. + +The `cluster` propety names a network-reachable backing cluster that speaks the Kafka procotol. It has the same schema as the `targetCluster` property of a virtual cluster. + +The existing virtual cluster schema will be modified to support top level `clusters` and to make use of `routers`. + +Specifically: + +* the existing `targetCluster` property will be made optional, and deprecated +* a new `cluster` property will support referencing a target cluster by name (using a distict property name seems slightly nicer than overloading the allowed type of the existing `targetCluster` to support `string` or `object`). +* support for new `router` property will be added. This is a reference to a router defined in `routerDefinitions`. +* exactly one of `router`, `cluster` or `targetCluster` will be required. +* `router` is mutually exclusive with `filters`. + +For example the old-style: + +```yaml +virtualClusters: + - name: my-vc + portIdentifiesNode: ... + filters: + ... + targetCluster: + bootstrapServer: ... + tls: ... +``` + +would be rewritten: + +```yaml +clusters: + - name: my-backing-cluster + bootstrapServer: ... + tls: ... +virtualClusters: + - name: my-vc + portIdentifiesNode: ... + filters: + ... + cluster: my-backing-cluster +``` + +An example of the `router` functionality: + +```yaml +clusters: + - name: my-backing-cluster + bootstrapServer: ... + tls: ... +routerDefinitions: + - name: my-router + type: MyRouter + config: + ... + routes: + - name: to-backing-cluster + filters: # a list of filter names + ... + cluster: my-backing-cluster +virtualClusters: + - name: my-vc + portIdentifiesNode: ... + + router: my-router +``` + +(note how the `filters` have moved from the virtual cluster to the route). + +There are some design choices inherent in the above rendering of the concepts into a configuration API. +Let's call some of them out explicitly: + +* The names of filter, router, and cluster definitions are each global to the configuration, but in their own namespace (e.g. a filter and a router may each be called 'foo' without this being ambiguous). +* A route is not a top-level entity, but belongs to a router. +* The names of routes must be unique within the scope of the containing router. +* A route may have filters in addition to a receiver. In this way a route embodies and generalizes the concept of a 'filter chain', which has never really been formalised in the proxy. + + +### Runtime + +**TODO** Api versions. All `ApiKeys`. +**TODO** Flow control & state machine. + + +### Metrics + +Routers would benefit dedicated metrics, implemented in the runtime. +They would be broadly similar to the existing metrics for Filters. + +## Affected/not affected projects + +The proxy. + +## Compatibility + +These changes would be fully backwards compatible: +* There would be no impact on the `Filter` API: All existing filters would continue to work. +* The changes to proxy configuration are backwards compatible. + +The choice to deprecate `targetCluster` in a virtual cluster, replacing it with `cluster` as a reference to a cluster defined at the top level, is made simply to try to reduce different ways of expressing the same configuration ("There should be one way to do it"). + +## Rejected alternatives + +* One alternative is simply to not do this (or not right now). +* `NetFilter` is an existing attempt at an abstraction for SASL Auth and cluster selection. It was never completed, and the interface never made it into the `kroxylicious-api` module. This proposal is more flexible since it allows routing decisions to happen after authentication. + From 55f72ee762b11714463df8a47543c7ff7b4d1d6b Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 16 Jun 2025 22:56:12 +0100 Subject: [PATCH 02/30] Apply suggestions from code review Co-authored-by: Gunnar Morling Signed-off-by: Tom Bentley Signed-off-by: Tom Bentley --- proposals/004-routing-api.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/proposals/004-routing-api.md b/proposals/004-routing-api.md index 9b131c75..11caabe7 100644 --- a/proposals/004-routing-api.md +++ b/proposals/004-routing-api.md @@ -21,7 +21,7 @@ Here are some examples: * **Union clusters**. Multiple backing Kafka clusters can be presented to clients as a single cluster. Broker-side entities, such as topics, get bijectively mapped (for example using a per-backing cluster prefix) to the -virtual entites presented to clients. +virtual entities presented to clients. `Filters` cannot easily do this because they're always hooked up to a single backing Kafka cluster. * **Topic splicing**. @@ -29,7 +29,7 @@ Multiple separate topics in distinct backing clusters are presented to clients a Only one backing topic is writable at any given logical time. * **Principal-aware routing**. -A natural variation on basic SASL termination is to use the identity of the authenticated client to drive the decision about which backing cluter to route requests to. +A natural variation on basic SASL termination is to use the identity of the authenticated client to drive the decision about which backing cluster to route requests to. Kroxylicious is currently unable to address use cases like these. @@ -56,7 +56,7 @@ Routers and backing clusters necessarily handle the whole protocol. Each `Router` implementation will support 0 or more named routes. The available and required route names will depend on the implementation, which might ascribe different behaviour to different named routes. For example a `Router` implementing the 'union cluster' use case might simply use the route names as prefixes for names of the broker-side entities -it will expose (such as topics or consumer groups), as as such impose no restriction on the supported route names. +it will expose (such as topics or consumer groups), and as such impose no restriction on the supported route names. In contrast, a `Router` implementing the 'topic splicing' use case might require configuration about each of the clusters being used in the splice, which would required the route names to be referenced in the `Router`'s configuration. @@ -134,8 +134,8 @@ Because routers can refer to other routers they form a graph. ... ``` -All _possible_ routes through the graph can be determined statically from the proxy configuration, but the routing of any individual incoming request is determined at runtime -can might involve multiple outgoing requests from any given router. +All _possible_ routes through the graph can be determined statically from the proxy configuration, but the routing of any individual incoming request is determined at runtime. +It may involve multiple outgoing requests to one or more clusters or routers. Validation performed at proxy startup will reject cyclic graphs. This will prevent the possibility of a request getting stuck in a router loop. From 8f0b19b1bcdcf02a94fe63d5d68b704a4b67b780 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Tue, 17 Jun 2025 10:00:55 +1200 Subject: [PATCH 03/30] Clarify wording Signed-off-by: Tom Bentley --- proposals/004-routing-api.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/proposals/004-routing-api.md b/proposals/004-routing-api.md index 11caabe7..9c657c63 100644 --- a/proposals/004-routing-api.md +++ b/proposals/004-routing-api.md @@ -139,9 +139,10 @@ It may involve multiple outgoing requests to one or more clusters or routers. Validation performed at proxy startup will reject cyclic graphs. This will prevent the possibility of a request getting stuck in a router loop. -In order for non-trivial router graphs to be useful, `Router` authors will need to follow the same principle of composition as `Filter` authors. -That is, a `Router` implementation should only talk to its receivers, and not, for example, make direct connections of its own to a backing cluster. -To do so would prevent use of that router implementation in a larger graph. +In order for non-trivial router graphs to be useful, `Router` authors will need to follow the same _principle of composition_ as `Filter` authors. +That is, a `Router` implementation should only talk to its receivers using the RouterContext API, and not, for example, make their own direct TCP connections to a backing cluster. +Doing so would shortcircuit any logic in upstream routers and filters, which could be manipulating broker-side entities like topic names. +Such shortcircuiting would prevents use of that router implementation in a larger graph. The `cluster` propety names a network-reachable backing cluster that speaks the Kafka procotol. It has the same schema as the `targetCluster` property of a virtual cluster. From 7e03111ed95a4da366336036cfc43fca6c3bea9a Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 22 May 2026 08:48:42 +1200 Subject: [PATCH 04/30] Rename routing proposal for new numbering scheme Signed-off-by: Tom Bentley --- proposals/{004-routing-api.md => 070-routing-api.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename proposals/{004-routing-api.md => 070-routing-api.md} (100%) diff --git a/proposals/004-routing-api.md b/proposals/070-routing-api.md similarity index 100% rename from proposals/004-routing-api.md rename to proposals/070-routing-api.md From 734c2455da8aadebd7da2f98e6e972f6903bcbf0 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 22 May 2026 08:52:15 +1200 Subject: [PATCH 05/30] Add number to title Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 9c657c63..f81380f9 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -1,6 +1,6 @@ -# A Routing API +# 70 - A Routing API This proposal discusses an API for routing requests to Kafka clusters. From a5f60d393fce15a419d9b3a96a5fc695ace48e23 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 22 May 2026 11:08:53 +1200 Subject: [PATCH 06/30] Start rework Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index f81380f9..d181e2ba 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -1,16 +1,14 @@ - - # 70 - A Routing API This proposal discusses an API for routing requests to Kafka clusters. ## Current situation -Kroxylicious currently supports `Filter` plugins as the top-level mechanism for adding behaviour to a proxy. +Kroxylicious currently supports `Filter` plugins as the top-level mechanism for adding behaviour to a proxy instance. `Filters` can manipulate Kafka protocol requests and responses being sent by/to the Kafka client. They can also originate requests of their own (for example to obtain metadata that's necessary for them to function). -However, `Filters` cannot influence which Kafka cluster will receive the requests which it forwards or makes for itself. +However, a `Filter` cannot influence which Kafka cluster will receive the requests it handles. ## Motivation @@ -22,17 +20,26 @@ Here are some examples: Multiple backing Kafka clusters can be presented to clients as a single cluster. Broker-side entities, such as topics, get bijectively mapped (for example using a per-backing cluster prefix) to the virtual entities presented to clients. +For databases, this is known as Data Virtualization. `Filters` cannot easily do this because they're always hooked up to a single backing Kafka cluster. +* **Principal-aware routing**. +A natural variation on basic [SASL termination](004-terminology-for-authentication.md) is to use the identity of the authenticated client to drive the decision about which backing cluster to route requests to. +This could be used with some metadata about principals to ensure that a client is routed to a cluster that is local to it. + * **Topic splicing**. Multiple separate topics in distinct backing clusters are presented to clients as a single topic. Only one backing topic is writable at any given logical time. - -* **Principal-aware routing**. -A natural variation on basic SASL termination is to use the identity of the authenticated client to drive the decision about which backing cluster to route requests to. Kroxylicious is currently unable to address use cases like these. +**Caveat:** It's worth noting that some aspects of the Kafka protocol prevents routing use cases which might, as first glance, appear possible. +An example is _Record-based routing_. +While it's possible to use some attribute of an individual record (for example a header) to determine the destination for a `PRODUCE` request, +problems arise when you consider how a router should handle the offsets in the `PRODUCE` response returned to the client. +While some client applications might not make use of record offsets, a proxy cannot make that assumption. +It would be possible to route record _batches_ without complication, but a record batch is not a first class concept in the Kafka Producer API, and lacks relevant (e.g. user-supplied) metadata for making routing decisions. + ## Proposal ### Concepts From 365fd19cc182b2abe737e116fb0b08f6b7e3c68b Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 22 May 2026 13:37:59 +1200 Subject: [PATCH 07/30] Big reworking Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 534 ++++++++++++++++++++++++++--------- 1 file changed, 408 insertions(+), 126 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index d181e2ba..ee905443 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -33,74 +33,228 @@ Only one backing topic is writable at any given logical time. Kroxylicious is currently unable to address use cases like these. -**Caveat:** It's worth noting that some aspects of the Kafka protocol prevents routing use cases which might, as first glance, appear possible. +**Caveat:** It's worth noting that some aspects of the Kafka protocol prevent routing use cases which might, at first glance, appear possible. An example is _Record-based routing_. While it's possible to use some attribute of an individual record (for example a header) to determine the destination for a `PRODUCE` request, problems arise when you consider how a router should handle the offsets in the `PRODUCE` response returned to the client. While some client applications might not make use of record offsets, a proxy cannot make that assumption. It would be possible to route record _batches_ without complication, but a record batch is not a first class concept in the Kafka Producer API, and lacks relevant (e.g. user-supplied) metadata for making routing decisions. +Not all use cases are equal in complexity. +Producer-side routing (deciding which cluster handles a `PRODUCE` request based on its topic) delivers the most immediate value, +and has the simplest protocol interactions. +Consumer-side operations, admin operations, and stateful protocol features (transactions, consumer groups, fetch sessions) each add successive layers of complexity. +The API proposed here is general-purpose and does not prescribe which protocol features a `Router` must support; +individual `Router` implementations are free to support as much or as little of the protocol as their use case requires. + ## Proposal ### Concepts -To enable the use cases above we need a few new concepts: +To enable the use cases above we need a few concepts: + +* A _cluster_ is an upstream Kafka cluster that can handle requests. +* A _router_ is a thing which decides which _route(s)_ should be used for a given request. +* A _route_ is a named pathway from a router towards a _target_. +* A route's _target_ is either a cluster or another router. Routes may also have filters attached. + +Because a route's target can be another router, routers and routes together form a directed graph. +Furtermore, we will disallow loops in the graph, so we have a directed acyclic graph (DAG). + +``` + route "foo" + .------[filters...]------> cluster-a + / + client --> virtual cluster --> router + (my-vc) \ + '------[filters...]------> cluster-b + route "bar" +``` + +The _virtual cluster_ is the entry point: it binds a client-facing network address to a target (here, a router). +The _router_ decides which _route_ each request should traverse. +Each _route_ may carry its own filter chain, and terminates at a _cluster_ (or another router). +Clusters are the leaf nodes of the graph. + +#### Virtual node IDs + +Kafka's protocol identifies brokers by integer _node IDs_. +These node IDs are scoped to a single cluster — broker 0 in cluster-a is a completely different machine from broker 0 in cluster-b. +When a router presents multiple clusters to the client as a single virtual cluster, there is a collision problem: the client might receive node ID 0 in a `METADATA` response from one route and node ID 0 from another, with no way to distinguish them. -* A _receiver_ is something that can handle requests, and which will return at most one response. -* A _route_ represents a possible pathway from an incoming request towards a receiver. -* A _router_ is a thing which decides which route should be used for a given request. +The solution is _virtual node IDs_. +The runtime assigns each `(route, target-cluster node ID)` pair a unique virtual node ID. +Virtual node IDs are `int32`, matching Kafka's wire format for node IDs. We believe this is large enough not to be a practical limit. +All responses delivered to the router (and onward to the client) use virtual node IDs. +All requests from the router use virtual node IDs. +The runtime transparently translates between virtual and real node IDs at the boundary. + +This means a router author works entirely in terms of virtual node IDs and never needs to translate. +When a `METADATA` response arrives with a list of brokers, the node IDs in that response are already virtual. +When the router wants to send a request to one of those brokers, it simply passes the virtual node ID back. +The runtime handles the mapping. + +The details of how the mapping works (the formula, the implementation) are described in the _Runtime_ section below. +The key point here is that virtual node IDs are the universal addressing scheme within the routing API. -Routers and backing Kafka clusters are both examples of receivers. -Slightly more generally, a receiver is anything that speaks the full Kafka procotol. -We don't consider a `Filter` to be a receiver, even though it can make short-circuit responses. -`Filters` usually rely on having a backing Kafka cluster to forward requests to. -Generally speaking, a `Filter` might only handle a subset of the `ApiKeys` of the Kafka protocol. -Routers and backing clusters necessarily handle the whole protocol. ### Plugin API -`Router` will be a top level plugin analogous to the `Filter` plugin interface, using the same `ServiceLoader`-based mechanism for runtime discovery. +#### `Router` + +`Router` is a top-level plugin analogous to the `Filter` plugin interface, using the same `ServiceLoader`-based mechanism for runtime discovery. + Each `Router` implementation will support 0 or more named routes. The available and required route names will depend on the implementation, which might ascribe different behaviour to different named routes. -For example a `Router` implementing the 'union cluster' use case might simply use the route names as prefixes for names of the broker-side entities +For example, a `Router` implementing the 'union cluster' use case might simply use the route names as prefixes for names of the broker-side entities it will expose (such as topics or consumer groups), and as such impose no restriction on the supported route names. -In contrast, a `Router` implementing the 'topic splicing' use case might require configuration about each of the clusters being used in the splice, which -would required the route names to be referenced in the `Router`'s configuration. +In contrast, a `Router` implementing the 'topic splicing' use case might require configuration about each of the clusters being used in the splice, which would require the route names to be referenced in the `Router`'s configuration. ```java interface Router { - CompletionStage onClientRequest(short apiVersion, - ApiKeys apiKey, - RequestHeaderData header, - ApiMessage request, - RoutingContext context); + CompletionStage onRequest(short apiVersion, + ApiKeys apiKey, + RequestHeaderData header, + ApiMessage request, + RouterContext context); + + default Map staticRoutes() { + return Map.of(); + } + + default void close() {} +} +``` + +A `Router` instance is created **per client connection**, not per virtual cluster. +This means per-connection state (caches, session state) can live directly in the `Router` instance without synchronisation, since all calls to a given instance happen on a single Netty event loop thread. +State shared across connections belongs in the `RouterFactory`'s initialisation data (see below). + +**`onRequest()`** is invoked for each incoming client request that is _dynamically routed_. +The router inspects the request, decides which route(s) to use, sends one or more requests via the `RouterContext`, and eventually delivers a response to the client. +The returned `CompletionStage` completes when the router has finished processing the request. + +**`staticRoutes()`** returns a map of `ApiKeys` to route names for requests that should always be forwarded to a fixed route without deserialisation. +This is a performance optimisation: the runtime can forward these requests as opaque frames, bypassing the cost of deserialisation (assuming no `Filters` require deserialization) and calling `onRequest()`. +API keys not present in the map are dynamically routed via `onRequest()`. +The default implementation returns an empty map (all requests dynamically routed). + +**`close()`** is called when the client connection is torn down. +Implementations should release any per-connection resources (session caches, etc.). + +#### `RouterFactory` + +`RouterFactory` manages the lifecycle of `Router` instances, analogous to `FilterFactory` for filters. + +```java +interface RouterFactory { + I initialize(RouterFactoryContext context, C config); + + Router createRouter(RouterFactoryContext context, I initializationData); + + void close(I initializationData); } ``` -For a given incoming request a `Router` implementation can decide which route(s) to make a request to. -We want to allow a router to potentially make multiple requests (e.g. to multiple clusters) and to have control over their processing (e.g. sequential or concurrent). -For this reason the `RoutingContext` does not follow the builder pattern used in the `FilterContext`, but simply -exposes methods to asynchronously send requests down a given route. -This allows the `Router` author to make use of the `CompletionStage` API when issuing multiple requests. +**`initialize()`** is called once per virtual cluster that uses this router. +The configuration type `C` is deserialized from the router's `config` property in the proxy configuration. +The returned initialisation data `I` is shared across all `Router` instances created for that virtual cluster. +Because `Router` instances may be created on different event loop threads, the initialisation data must be thread-safe. + +**`createRouter()`** is called once per client connection. It may run on a different thread than `initialize()`. + +**`close(I)`** is called when the virtual cluster shuts down, to release shared resources. It may run on a different thread than `initialize()`. + +**`RouterFactoryContext`** provides context to the factory: +* `virtualClusterName()` — the name of the virtual cluster. +* `routerName()` — the name of this router within the configuration. +* `pluginInstance()` / `pluginImplementationNames()` — access to the plugin registry. + +#### `RouterContext` + +`RouterContext` is passed to `Router.onRequest()` and provides methods for issuing requests to routes and delivering responses to the client. ```java -interface RoutingContext { +interface RouterContext { + + int bootstrapNodeId(String route); + + CompletionStage sendRequestToNode(int virtualNodeId, + RequestHeaderData header, + ApiMessage request); - CompletionStage sendRequest(String route, ...) - void sendResponse(Resonse) - void disconnect() + String sessionId(); + Subject authenticatedSubject(); } ``` +We want to allow (but not require) a router to potentially make multiple requests (e.g. to multiple clusters) and to have control over their processing (e.g. sequential or concurrent). +For this reason `RouterContext` does not follow the builder pattern used in the `FilterContext`, but simply exposes methods to asynchronously send requests. +This allows the `Router` author to make use of the `CompletionStage` API when issuing multiple requests. + +**`bootstrapNodeId(route)`** returns the virtual node ID of the bootstrap broker for a named route. +This is used to send the initial requests (e.g. `METADATA`) before the router has discovered the cluster's broker topology. +Once `METADATA` responses arrive, the router uses the virtual node IDs from those responses to address specific brokers — no translation is needed, since the node IDs in responses are already virtual (see _Virtual node IDs_ above). + +**`sendRequestToNode(virtualNodeId, ...)`** sends a request to a specific broker, identified by its virtual node ID. +The runtime resolves the virtual ID to a route and upstream broker address, opening a new connection if necessary. + +All requests are addressed by virtual node ID. There is no "send to any broker" method. +This is a deliberate design choice: the Kafka protocol scopes most operations to a specific broker (partition leaders, group coordinators, per-broker `API_VERSIONS` negotiation). +Providing a "send to any broker" convenience method would be a footgun for router authors — it would be easy to use for an API that actually requires a specific broker, and the resulting bug would only manifest at runtime under specific conditions (e.g. multi-broker clusters, rolling upgrades). +By requiring a virtual node ID for every request, the router author is forced to think about broker targeting, which correctly reflects the protocol's reality. + +**`sessionId()`** and **`authenticatedSubject()`** provide observability and identity context. +`authenticatedSubject()` returns the `Subject` established by upstream SASL processing (e.g. a SASL termination filter). +If no SASL processing has occurred, the subject will be anonymous. +Correct placement of SASL plugins in the topology is the operator's responsibility; +a future proposal may add runtime validation of SASL plugin placement. + +All `RouterContext` methods are called on the same Netty event loop thread as `onRequest()`. +No synchronisation is needed within a single `Router` instance. + +#### `RouterResult` + +`RouterResult` is the return type from `onRequest()`. +It is a sealed type that encodes the outcome of request processing: + +```java +sealed interface RouterResult { + record Completed(Response response) implements RouterResult {} + record CompletedNoResponse() implements RouterResult {} + record Disconnect() implements RouterResult {} +} +``` + +**`Completed(response)`** — the router has produced a response for the client. +The runtime delivers the response, automatically rewriting the correlation ID to match the client's original request. + +**`CompletedNoResponse()`** — the router has finished processing but there is no response to deliver. +This is the correct result for fire-and-forget requests (e.g. `PRODUCE` with `acks=0`). +Having a dedicated type rather than a nullable response field makes the `acks=0` case explicit. +The runtime can log a warning if a router returns `Completed` for a request that has no response, or `CompletedNoResponse` for a request that expects one. + +**`Disconnect()`** — the router requests that the client connection be closed. + +This follows the pattern established by `FilterResult`, where the outcome is a value returned to the runtime rather than an imperative side-effect on the context. +The sealed hierarchy can be extended with new variants (e.g. `CompletedThenDisconnect(Response)`) in future without breaking existing implementations. + +If the `CompletionStage` completes exceptionally, the runtime treats this as an unrecoverable error and closes the client connection. + ### Configuration -Routers are configured at the top level of the proxy configuration, similarly to `filterDefinitions`: -In addition to the `name`, `type` and `config` (which serve the same purpose for `Routers` as they do for `Filters`), a `routerDefinition` also supports a `routes` property. -The `routes` property is optional, though any given implementation may have its own particular requirements for its `routes`. +Routers are configured at the top level of the proxy configuration, alongside cluster definitions and virtual clusters. ```yaml +clusterDefinitions: + - name: cluster-a + bootstrapServers: kafka-a:9092 + tls: ... + - name: cluster-b + bootstrapServers: kafka-b:9092 + routerDefinitions: - name: my-router type: MyRouter @@ -110,147 +264,275 @@ routerDefinitions: filters: - my-first-filter - my-second-filter - cluster: my-backing-cluster + target: + cluster: cluster-a - name: bar filters: - my-third-filter - router: my-other-router + target: + router: my-other-router - name: my-other-router - # ... + type: AnotherRouter + config: ... + routes: + - name: baz + target: + cluster: cluster-b + +virtualClusters: + - name: my-vc + target: + router: my-router + gateways: [...] + filters: + - my-audit-filter ``` -A route object has a `name`, an optional list of `filters` (being the names of the filters to be applied to requests/responses that traverse this route) and either a -`cluster` or a `router` property, which names the receiver which will handle requests after any filters have been applied. -Exactly one of `cluster` or `router` must be specified. +The proxy configuration has three top-level definition lists: -Because routers can refer to other routers they form a graph. +* **`clusterDefinitions`** — the catalogue of upstream Kafka clusters the proxy can connect to. Each has a `name`, `bootstrapServers`, and optional `tls` configuration. -``` - my-backing-cluster - ^ - / - / foo - requests / - ---------> my-router - \ - \ bar - \ ... - v / - my-other-router --- ... - \ - ... -``` +* **`routerDefinitions`** — router plugin instances. Each has a `name`, `type` (the plugin implementation name), optional `config`, and `routes`. In addition to the `name`, `type` and `config` (which serve the same purpose for routers as they do for filters), a router definition also supports a `routes` property. The `routes` property is optional, though any given implementation may have its own particular requirements for its routes. -All _possible_ routes through the graph can be determined statically from the proxy configuration, but the routing of any individual incoming request is determined at runtime. -It may involve multiple outgoing requests to one or more clusters or routers. -Validation performed at proxy startup will reject cyclic graphs. -This will prevent the possibility of a request getting stuck in a router loop. +* **`virtualClusters`** — the entry points presented to clients. Each has a `name`, `gateways`, and a `target`. -In order for non-trivial router graphs to be useful, `Router` authors will need to follow the same _principle of composition_ as `Filter` authors. -That is, a `Router` implementation should only talk to its receivers using the RouterContext API, and not, for example, make their own direct TCP connections to a backing cluster. -Doing so would shortcircuit any logic in upstream routers and filters, which could be manipulating broker-side entities like topic names. -Such shortcircuiting would prevents use of that router implementation in a larger graph. +#### Routes + +A route has a `name`, an optional list of `filters` (the names of filters applied to requests/responses traversing this route), and a `target`. -The `cluster` propety names a network-reachable backing cluster that speaks the Kafka procotol. It has the same schema as the `targetCluster` property of a virtual cluster. +The `target` property is a discriminated union containing exactly one of: +* `cluster` — the name of a cluster defined in `clusterDefinitions`. +* `router` — the name of another router defined in `routerDefinitions`. -The existing virtual cluster schema will be modified to support top level `clusters` and to make use of `routers`. +This `target` structure is used uniformly on both routes and virtual clusters, providing a consistent way to express "what does this connect to" throughout the configuration. -Specifically: +#### Virtual cluster changes -* the existing `targetCluster` property will be made optional, and deprecated -* a new `cluster` property will support referencing a target cluster by name (using a distict property name seems slightly nicer than overloading the allowed type of the existing `targetCluster` to support `string` or `object`). -* support for new `router` property will be added. This is a reference to a router defined in `routerDefinitions`. -* exactly one of `router`, `cluster` or `targetCluster` will be required. -* `router` is mutually exclusive with `filters`. +The existing virtual cluster schema is modified: +* The existing inline `targetCluster` property is deprecated but still supported for backwards compatibility. It will be removed in a future release. +* A new `target` property supports referencing a cluster or router by name (as described above). +* Exactly one of `target` or `targetCluster` must be specified. +* Virtual clusters may have both `filters` and a `target` referencing a router. The virtual cluster's filters run before router dispatch, handling cross-cutting concerns such as audit logging or authorisation that apply regardless of the routing decision. -For example the old-style: +For example, the old-style configuration: ```yaml virtualClusters: - name: my-vc - portIdentifiesNode: ... - filters: - ... + gateways: [...] + filters: [...] targetCluster: - bootstrapServer: ... - tls: ... + bootstrapServers: kafka:9092 ``` -would be rewritten: +can be rewritten as: ```yaml -clusters: - - name: my-backing-cluster - bootstrapServer: ... - tls: ... +clusterDefinitions: + - name: my-cluster + bootstrapServers: kafka:9092 + virtualClusters: - name: my-vc - portIdentifiesNode: ... - filters: - ... - cluster: my-backing-cluster -``` + gateways: [...] + filters: [...] + target: + cluster: my-cluster +``` -An example of the `router` functionality: +#### Router graph -```yaml -clusters: - - name: my-backing-cluster - bootstrapServer: ... - tls: ... -routerDefinitions: - - name: my-router - type: MyRouter - config: - ... - routes: - - name: to-backing-cluster - filters: # a list of filter names - ... - cluster: my-backing-cluster -virtualClusters: - - name: my-vc - portIdentifiesNode: ... +As mentioned earlier, because routers can refer to other routers via their routes, they form a directed graph. +All possible routes through the graph can be determined statically from the proxy configuration, but the routing of any individual incoming request is determined at runtime. +It may involve multiple outgoing requests to one or more clusters or routers. - router: my-router -``` +Validation performed at proxy startup will reject: +* Cyclic graphs (preventing request loops). +* Dangling references (routes or virtual clusters referencing undefined clusters or routers). -(note how the `filters` have moved from the virtual cluster to the route). +In order for non-trivial router graphs to be useful, `Router` authors will need to follow the same _principle of composition_ as `Filter` authors. +That is, a `Router` implementation should only talk to its targets using the `RouterContext` API, and not, for example, make their own direct TCP connections to a backing cluster. +Doing so would short-circuit any logic in downstream routers and filters, which could be manipulating broker-side entities like topic names. -There are some design choices inherent in the above rendering of the concepts into a configuration API. -Let's call some of them out explicitly: +### Runtime -* The names of filter, router, and cluster definitions are each global to the configuration, but in their own namespace (e.g. a filter and a router may each be called 'foo' without this being ambiguous). -* A route is not a top-level entity, but belongs to a router. -* The names of routes must be unique within the scope of the containing router. -* A route may have filters in addition to a receiver. In this way a route embodies and generalizes the concept of a 'filter chain', which has never really been formalised in the proxy. +#### Threading model +One `Router` instance exists per client connection, running on a single Netty event loop thread. +All `onRequest()` invocations and `RouterContext` method calls for that instance happen on this thread. +No synchronisation is needed within a `Router` instance. -### Runtime +`RouterFactory.initialize()` may run on a different thread than `createRouter()`. +Any initialisation data shared across connections must be thread-safe. + +#### Request dispatch + +The runtime dispatches incoming requests in one of two modes: + +* **Static routes**: For API keys declared via `Router.staticRoutes()`, the runtime forwards the request as an opaque frame directly to the named route's backend, bypassing deserialisation and `Router.onRequest()`. This is a performance optimisation for APIs that the router does not need to inspect. + +* **Dynamic routes**: For all other API keys, the runtime deserialises the request, creates a `RouterContext`, and invokes `Router.onRequest()`. The router uses the context to send requests to routes and deliver a response to the client. + +#### Correlation ID management + +When a router sends requests via `RouterContext`, the runtime allocates _routing correlation IDs_ that are distinct from the client's original correlation IDs. +Routing correlation IDs are negative integers (allocated from `Integer.MIN_VALUE / 2` upward), which distinguishes them from client-originated correlation IDs (which are non-negative). + +When the router returns a `Completed(response)` result, the runtime automatically rewrites the response header's correlation ID to match the client's original request. +Routers never need to manage correlation IDs themselves. + +#### Response ordering + +A router may send multiple requests (fan-out) and compose their responses before delivering a single response to the client. +Different routes may respond at different times. +The runtime uses a _response sequencer_ to ensure that responses are delivered to the client in the same order as the original client requests, regardless of the order in which fan-out responses arrive. + +#### Node ID mapping implementation + +As described in _Virtual node IDs_ above, the runtime translates between real target-cluster node IDs and virtual node IDs. +This is implemented by a `NodeIdMapping` abstraction (internal to the runtime) with two operations: + +* `toVirtual(route, targetNodeId)` — used when rewriting responses received from a target cluster (the runtime rewrites broker node IDs in `METADATA`, `PRODUCE`, `FIND_COORDINATOR`, and `DESCRIBE_CLUSTER` responses before they reach the router). +* `fromVirtual(virtualNodeId)` — used when the router calls `sendRequestToNode()`, returning both the route name and the target-cluster node ID. + +The mapping has a strict invertibility invariant: `fromVirtual(toVirtual(route, t))` must return `(route, t)`. +This means a virtual node ID always unambiguously identifies both the route and the target-cluster broker. + +The current implementation uses a bijective mapping with the formula: + +``` +V = offset + N × t +``` + +where `V` is the virtual node ID, `t` is the target-cluster node ID, `N` is the number of routes, and `offset` is the route's zero-based index. +This mapping is deterministic, stateless, and O(1) in both directions. +No coordination between proxy instances is required. + +The bijective mapping is _stable_: adding or removing brokers from a target cluster does not change the virtual node IDs of existing brokers. +Likewise, adding or removing proxy instances has no effect on the mapping. +This stability is important because Kafka clients cache broker metadata and will use previously-seen node IDs in subsequent requests. + +For single-route configurations, an identity mapping is used: the virtual ID equals the target ID, with zero overhead. -**TODO** Api versions. All `ApiKeys`. -**TODO** Flow control & state machine. +All proxy instances presenting the same virtual cluster to clients must use the same `NodeIdMapping`. +Changing the mapping strategy (e.g. from bijective to a different scheme) is expected to be a disruptive operation — not a zero-downtime change — because connected clients will hold stale virtual node IDs from the previous mapping. + +In future, if proxy instances were themselves presented to clients as broker nodes (e.g. in a clustered proxy deployment), the mapping would need to account for the proxy's own identity. +Such a mapping would require strong consistency across proxy instances to ensure they agree on the virtual node ID assignment. +The `NodeIdMapping` abstraction accommodates this evolution without changing the `Router` API. + +`NodeIdMapping` is a `sealed` interface — the runtime controls which implementations exist. +`Router` authors never implement or interact with this interface directly. + +#### API version negotiation + +The Kafka protocol scopes `API_VERSIONS` to a single broker connection — a client sends `API_VERSIONS` each time it connects to a new broker. +This means the proxy must forward `API_VERSIONS` to the specific broker the client believes it is talking to, not to an arbitrary broker. + +With `BijectiveNodeIdMapping`, each virtual node ID maps to exactly one real broker, so the router can use `sendRequestToNode()` to forward `API_VERSIONS` to the correct broker. +With a `NodeIdMapping` where virtual IDs do not correspond to unique real brokers, the runtime would need to fan out `API_VERSIONS` to all brokers behind that virtual ID and return the intersection of their version ranges. + +The runtime's existing `ApiVersionsIntersectFilter` intersects each backend broker's version ranges with the proxy's own maximums. + +Routers may choose to _further constrain_ advertised API versions. +For example, a router that fans out requests by topic name might cap `PRODUCE` to v12 to avoid handling topic IDs (which are cluster-specific in v13+), though this is an implementation simplification rather than a fundamental necessity. +Version capping is the router's responsibility, not the runtime's. + +#### Flow control + +The runtime uses Netty's auto-read mechanism to apply TCP-level backpressure to the client connection. +Auto-read is disabled while the router is processing a request and re-enabled when the `CompletionStage` completes. +This means the router's processing strategy directly determines the proxy's response latency and backpressure behaviour: + +* A router that forwards a request to a single backend and returns the response has latency equal to the backend's latency. +* A router that fans out to multiple backends and waits for all responses (e.g. `CompletableFuture.allOf(...)`) has latency equal to the _maximum_ of the backends' latencies. A slow backend will hold up the client. +* A router that fans out but completes as soon as one backend responds (e.g. for speculative execution) has latency equal to the _minimum_. + +The runtime does not impose its own timeout on router processing. +If a router blocks indefinitely (e.g. a backend never responds), the client connection will stall. +Routers are responsible for implementing their own timeouts if needed. + +Requests arriving before backend connections are ready are buffered and flushed when the connection becomes active. +Backpressure from any upstream connection propagates to the client channel. ### Metrics -Routers would benefit dedicated metrics, implemented in the runtime. -They would be broadly similar to the existing metrics for Filters. +The runtime provides the following per-route metrics: + +* Request counters (total, errors) tagged by route name, API key, and routing mode (static/dynamic). +* Request latency histograms tagged by route name and API key. +* Error counters tagged by error type (unknown route, node forward failure, router failure). + +These are broadly similar to the existing per-filter metrics. + +#### Cardinality analysis + +The metric tag dimensions determine the number of distinct time series. Operators and monitoring systems need to be able to handle the resulting cardinality. + +| Tag | Typical values | Bound | +|-----|---------------|-------| +| route name | 2–5 per router | Bounded by configuration (number of routes). | +| API key | ~80 defined in the Kafka protocol, but only ~20 are common in practice | Bounded by the protocol. Grows slowly (a few new keys per Kafka release). | +| routing mode | `static`, `dynamic` | Fixed at 2. | +| error type | `unknown_route`, `node_forward_failed`, `router_failed` | Fixed at 3. | + +The routing mode is determined per API key (an API key is either statically or dynamically routed, never both), so it does not multiply the cardinality — it is a property of the key, not an independent dimension. + +For the request counter, the worst-case cardinality per virtual cluster is `routes × API keys`. With 3 routes and 80 API keys, that is 240 series. In practice it will be much lower because most API keys are statically routed (producing one series per key, not per route) and many API keys are never used by a given client. + +For the latency histogram, the cardinality is `routes × API keys`. With 3 routes and 80 API keys, that is 240 series. Again, in practice most of these will never be populated. + +For the error counter, the cardinality is bounded by the 3 error types — negligible. + +This cardinality is comparable to the existing per-filter metrics and should not pose problems for standard monitoring deployments. We deliberately avoid high-cardinality tags such as client ID, topic name, or session ID. + ## Affected/not affected projects -The proxy. +* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResult`, `Response`. +* `kroxylicious-runtime` — routing engine, configuration model (`RouterDefinition`, `RouteDefinition`, `TargetClusterDefinition`), graph validation, node ID mapping, response sequencing, metrics. +* `kroxylicious-bom` — version management for any new modules. +* Not affected: existing filters, KMS, authoriser API. The Kubernetes operator will need a separate update to support router configuration in CRDs. + ## Compatibility -These changes would be fully backwards compatible: -* There would be no impact on the `Filter` API: All existing filters would continue to work. -* The changes to proxy configuration are backwards compatible. +These changes are fully backwards compatible: +* There is no impact on the `Filter` API. All existing filters continue to work. +* The inline `targetCluster` property on virtual clusters is deprecated but still supported. Existing configurations continue to work without changes. +* The new `clusterDefinitions`, `routerDefinitions`, and `target` properties are purely additive. + +The deprecation of inline `targetCluster`, replacing it with a named cluster reference via `target: { cluster: }`, is made to reduce different ways of expressing the same configuration. +Having a single `clusterDefinitions` list as the authoritative catalogue of upstream clusters makes the proxy's connectivity easy to audit. -The choice to deprecate `targetCluster` in a virtual cluster, replacing it with `cluster` as a reference to a cluster defined at the top level, is made simply to try to reduce different ways of expressing the same configuration ("There should be one way to do it"). ## Rejected alternatives -* One alternative is simply to not do this (or not right now). -* `NetFilter` is an existing attempt at an abstraction for SASL Auth and cluster selection. It was never completed, and the interface never made it into the `kroxylicious-api` module. This proposal is more flexible since it allows routing decisions to happen after authentication. +* **Do nothing.** One alternative is simply not to add routing (or not right now). However, the use cases described in the motivation are real and cannot be addressed with the `Filter` API alone. + +* **`NetFilter`.** `NetFilter` is an existing attempt at an abstraction for SASL auth and cluster selection. It was never completed, and the interface never made it into the `kroxylicious-api` module. This proposal is more flexible since it allows routing decisions to happen after authentication. + +* **Top-level route definitions.** We considered making routes first-class top-level objects with their own definition list. This was rejected because a route's identity is inherently scoped to its parent router — the same route name in different routers means different things. Making routes top-level would require a fourth definition list and introduce potential for confusion. + +* **Uniform `nodes` list with `kind` discriminator.** We considered a single polymorphic list for all graph nodes (virtual clusters, routers, clusters) with a `kind` field to distinguish them. This was rejected because typed lists are more readable, and virtual clusters, routers, and clusters serve fundamentally different roles (ingress, logic, egress). Having separate lists makes it immediately obvious where to look for each type. + +* **`connectsTo` / `receiver` property name.** We considered a verb-based property name (`connectsTo`) for the target reference, as well as the abstract noun `receiver` to unify routers and clusters. We settled on `target` as a simpler noun, with explicit type discrimination (`cluster` or `router`) inside the object. This is more concrete than `receiver` and more conventional than a verb. + +* **Mutual exclusivity of filters and router on a virtual cluster.** The original draft of this proposal required that a virtual cluster specify either `filters` or a `router`, but not both. In practice, virtual-cluster-level filters handle cross-cutting concerns (audit logging, authorisation) that apply regardless of the routing decision. The implementation allows both, with filters running before router dispatch. + + +## Design choices + +This section summarises the key design choices made in this proposal, for ease of reference. +* **Virtual node IDs** provide a uniform addressing scheme that insulates router authors from the node ID collision problem inherent in multi-cluster topologies. The runtime owns the mapping; routers work exclusively with virtual IDs and never need to translate. +* **Per-connection `Router` instances** allow per-connection state (caches, sessions) without synchronisation. Shared state lives in the `RouterFactory`'s initialisation data, which must be thread-safe. +* **`RouterResult` as a sealed type** encodes request outcomes (response, no-response, disconnect) as values rather than side-effects, following the pattern established by `FilterResult`. The sealed hierarchy is extensible without breaking existing implementations. +* **No "send to any broker" method.** All requests are addressed by virtual node ID. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. +* **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. +* **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. +* **Routes generalise filter chains.** A route may carry its own filter list in addition to a target, allowing different filter chains for different paths through the graph. +* **Route-level SASL** (e.g. SASL initiator for upstream authentication) is achieved via per-route filters rather than a dedicated route-level property. +* **Definition names** are global within their type (filters, routers, clusters each have their own namespace) but not across types. +* **Version capping is the router's responsibility**, not the runtime's. The runtime provides the baseline intersection of proxy and backend version ranges; routers may further constrain as needed. From 2acceef1f7f3a0bf216eccc8c03418e1de6fa7b2 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 22 May 2026 16:01:09 +1200 Subject: [PATCH 08/30] Tweak following review of the router impl proposal Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index ee905443..e4991ce2 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -193,10 +193,14 @@ We want to allow (but not require) a router to potentially make multiple request For this reason `RouterContext` does not follow the builder pattern used in the `FilterContext`, but simply exposes methods to asynchronously send requests. This allows the `Router` author to make use of the `CompletionStage` API when issuing multiple requests. -**`bootstrapNodeId(route)`** returns the virtual node ID of the bootstrap broker for a named route. -This is used to send the initial requests (e.g. `METADATA`) before the router has discovered the cluster's broker topology. +**`bootstrapNodeId(route)`** returns the virtual node ID of a broker on the named route's cluster. +This is used to send the initial requests (e.g. `METADATA`) before the router has discovered the cluster's full broker topology. Once `METADATA` responses arrive, the router uses the virtual node IDs from those responses to address specific brokers — no translation is needed, since the node IDs in responses are already virtual (see _Virtual node IDs_ above). +The runtime is responsible for selecting which broker to return. +A route's cluster may be configured with multiple bootstrap servers, and the runtime should randomise selection and round-robin on subsequent calls, mirroring how Kafka clients handle bootstrap addresses. +This keeps the selection policy in the runtime, where it can be applied consistently, rather than requiring each router to implement its own strategy. + **`sendRequestToNode(virtualNodeId, ...)`** sends a request to a specific broker, identified by its virtual node ID. The runtime resolves the virtual ID to a route and upstream broker address, opening a new connection if necessary. From 28e43a24c9e038e0721315698942b5b27fbadec2 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 25 May 2026 11:52:43 +1200 Subject: [PATCH 09/30] Add route to sendRequestToNode() to avoid locking in `BijectiveNodeIdMapping` assumptions Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 77 +++++++++++++++++++++++++----------- 1 file changed, 55 insertions(+), 22 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index e4991ce2..6d3c81f2 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -179,7 +179,8 @@ interface RouterContext { int bootstrapNodeId(String route); - CompletionStage sendRequestToNode(int virtualNodeId, + CompletionStage sendRequestToNode(String route, + int virtualNodeId, RequestHeaderData header, ApiMessage request); @@ -201,10 +202,18 @@ The runtime is responsible for selecting which broker to return. A route's cluster may be configured with multiple bootstrap servers, and the runtime should randomise selection and round-robin on subsequent calls, mirroring how Kafka clients handle bootstrap addresses. This keeps the selection policy in the runtime, where it can be applied consistently, rather than requiring each router to implement its own strategy. -**`sendRequestToNode(virtualNodeId, ...)`** sends a request to a specific broker, identified by its virtual node ID. -The runtime resolves the virtual ID to a route and upstream broker address, opening a new connection if necessary. +**`sendRequestToNode(route, virtualNodeId, ...)`** sends a request to a specific broker, identified by the route and a virtual node ID. +The runtime uses the route to determine which target cluster, and resolves the virtual node ID to a specific upstream broker address, opening a new connection if necessary. -All requests are addressed by virtual node ID. There is no "send to any broker" method. +The router passes both the route (which it knows from its routing decision) and the virtual node ID (which it knows from METADATA or `bootstrapNodeId()`). +Both pieces of information are naturally available at every call site — the route is the fundamental output of the routing decision, and the virtual node ID comes from cached metadata. + +The route parameter is essential for supporting different `NodeIdMapping` strategies (see _Node ID mapping implementation_ below). +With a dedicated (one-to-one) mapping, each virtual node belongs to exactly one route, so the route is redundant but serves as a cross-check. +With a shared (many-to-one) mapping — for example, if proxy instances were presented to clients as broker nodes — a single virtual node may serve brokers from multiple routes, and the route parameter is the only way for the runtime to determine which target cluster the request should reach. +Including the route in all cases avoids locking router implementations into a specific mapping strategy: the same router code works regardless of which mapping the runtime uses. + +There is no "send to any broker" method. This is a deliberate design choice: the Kafka protocol scopes most operations to a specific broker (partition leaders, group coordinators, per-broker `API_VERSIONS` negotiation). Providing a "send to any broker" convenience method would be a footgun for router authors — it would be easy to use for an API that actually requires a specific broker, and the resulting bug would only manifest at runtime under specific conditions (e.g. multi-broker clusters, rolling upgrades). By requiring a virtual node ID for every request, the router author is forced to think about broker targeting, which correctly reflects the protocol's reality. @@ -397,33 +406,56 @@ As described in _Virtual node IDs_ above, the runtime translates between real ta This is implemented by a `NodeIdMapping` abstraction (internal to the runtime) with two operations: * `toVirtual(route, targetNodeId)` — used when rewriting responses received from a target cluster (the runtime rewrites broker node IDs in `METADATA`, `PRODUCE`, `FIND_COORDINATOR`, and `DESCRIBE_CLUSTER` responses before they reach the router). -* `fromVirtual(virtualNodeId)` — used when the router calls `sendRequestToNode()`, returning both the route name and the target-cluster node ID. +* `fromVirtual(route, virtualNodeId)` — used when the router calls `sendRequestToNode(route, virtualNodeId, ...)`, returning the target-cluster node ID. -The mapping has a strict invertibility invariant: `fromVirtual(toVirtual(route, t))` must return `(route, t)`. -This means a virtual node ID always unambiguously identifies both the route and the target-cluster broker. +The mapping must satisfy: `fromVirtual(route, toVirtual(route, t))` returns `t` for any route and target node ID `t`. +Note that the mapping does not need to recover the route from the virtual node ID alone — the route is always supplied by the router. +This allows both _dedicated_ (one-to-one) and _shared_ (many-to-one) mapping strategies. -The current implementation uses a bijective mapping with the formula: +##### Mapping strategies -``` -V = offset + N × t -``` +The design supports several mapping strategies, each suited to different deployment topologies. -where `V` is the virtual node ID, `t` is the target-cluster node ID, `N` is the number of routes, and `offset` is the route's zero-based index. +**Dedicated mapping (one-to-one).** +Each virtual node maps to exactly one `(route, target-broker)` pair. +The current implementation uses a formula: `V = offset + N × t`, where `V` is the virtual node ID, `t` is the target-cluster node ID, `N` is the number of routes, and `offset` is the route's zero-based index. This mapping is deterministic, stateless, and O(1) in both directions. No coordination between proxy instances is required. -The bijective mapping is _stable_: adding or removing brokers from a target cluster does not change the virtual node IDs of existing brokers. +The client sees all brokers from all clusters (each with a unique virtual ID). +Because each virtual node belongs to exactly one route, leader-directed requests (e.g. `PRODUCE`, `FETCH`) from a well-behaved client will only contain topics from one route — no decomposition is needed for these APIs. +The route parameter in `sendRequestToNode()` is redundant here but serves as a cross-check. + +The dedicated mapping is _stable_: adding or removing brokers from a target cluster does not change the virtual node IDs of existing brokers. Likewise, adding or removing proxy instances has no effect on the mapping. This stability is important because Kafka clients cache broker metadata and will use previously-seen node IDs in subsequent requests. -For single-route configurations, an identity mapping is used: the virtual ID equals the target ID, with zero overhead. +For single-route configurations, an identity mapping (a special case of dedicated mapping) is used: the virtual ID equals the target ID, with zero overhead. -All proxy instances presenting the same virtual cluster to clients must use the same `NodeIdMapping`. -Changing the mapping strategy (e.g. from bijective to a different scheme) is expected to be a disruptive operation — not a zero-downtime change — because connected clients will hold stale virtual node IDs from the previous mapping. +**Shared mapping (many-to-one).** +Multiple `(route, target-broker)` pairs map to the same virtual node. +For example, if proxy instances are presented to clients as broker nodes, each proxy instance handles brokers from multiple routes. +The client sees fewer nodes (one per proxy instance rather than one per target broker). -In future, if proxy instances were themselves presented to clients as broker nodes (e.g. in a clustered proxy deployment), the mapping would need to account for the proxy's own identity. -Such a mapping would require strong consistency across proxy instances to ensure they agree on the virtual node ID assignment. -The `NodeIdMapping` abstraction accommodates this evolution without changing the `Router` API. +Because a single virtual node may serve brokers from multiple routes, a leader-directed request to one virtual node _can_ contain topics from different routes — decomposition is needed even for `PRODUCE` and `FETCH`. +The route parameter in `sendRequestToNode()` is essential here: without it, the runtime cannot determine which target cluster the request should reach. + +A shared mapping requires the proxy instances to agree on their assignments. +If assignments change dynamically (e.g. a proxy instance leaves or joins), a router may hold stale metadata — its cached leader for a topic might reference a virtual node that no longer serves that route. +This is not a programming error; it is a natural consequence of eventual consistency, analogous to Kafka's `NOT_LEADER_OR_FOLLOWER`. +The runtime should respond with an appropriate error, triggering the router to refresh its metadata. + +A shared mapping would require strong consistency across proxy instances to ensure they agree on the virtual node ID assignments. + +**Role-based mapping.** +Virtual nodes correspond to per-route functional roles (e.g. "partition leader", "group coordinator"). +If roles are scoped per route, this behaves like a dedicated mapping. +If roles span routes, it behaves like a shared mapping. + +##### Operational considerations + +All proxy instances presenting the same virtual cluster to clients must use the same `NodeIdMapping`. +Changing the mapping strategy (e.g. from dedicated to shared) is expected to be a disruptive operation — not a zero-downtime change — because connected clients will hold stale virtual node IDs from the previous mapping. `NodeIdMapping` is a `sealed` interface — the runtime controls which implementations exist. `Router` authors never implement or interact with this interface directly. @@ -433,8 +465,8 @@ The `NodeIdMapping` abstraction accommodates this evolution without changing the The Kafka protocol scopes `API_VERSIONS` to a single broker connection — a client sends `API_VERSIONS` each time it connects to a new broker. This means the proxy must forward `API_VERSIONS` to the specific broker the client believes it is talking to, not to an arbitrary broker. -With `BijectiveNodeIdMapping`, each virtual node ID maps to exactly one real broker, so the router can use `sendRequestToNode()` to forward `API_VERSIONS` to the correct broker. -With a `NodeIdMapping` where virtual IDs do not correspond to unique real brokers, the runtime would need to fan out `API_VERSIONS` to all brokers behind that virtual ID and return the intersection of their version ranges. +With a dedicated mapping, each virtual node ID maps to exactly one real broker, so the router can use `sendRequestToNode()` to forward `API_VERSIONS` to the correct broker. +With a shared mapping, the runtime would need to fan out `API_VERSIONS` to all brokers behind that virtual node and return the intersection of their version ranges. The runtime's existing `ApiVersionsIntersectFilter` intersects each backend broker's version ranges with the proxy's own maximums. @@ -531,9 +563,10 @@ Having a single `clusterDefinitions` list as the authoritative catalogue of upst This section summarises the key design choices made in this proposal, for ease of reference. * **Virtual node IDs** provide a uniform addressing scheme that insulates router authors from the node ID collision problem inherent in multi-cluster topologies. The runtime owns the mapping; routers work exclusively with virtual IDs and never need to translate. +* **Route included in `sendRequestToNode`** to support both dedicated (one-to-one) and shared (many-to-one) node ID mappings. With a dedicated mapping the route is redundant; with a shared mapping it is essential. Including it in all cases avoids locking router implementations into a specific mapping strategy, allowing the same router code to work across deployment topologies (e.g. each-broker-is-a-node vs proxy-instances-as-nodes). * **Per-connection `Router` instances** allow per-connection state (caches, sessions) without synchronisation. Shared state lives in the `RouterFactory`'s initialisation data, which must be thread-safe. * **`RouterResult` as a sealed type** encodes request outcomes (response, no-response, disconnect) as values rather than side-effects, following the pattern established by `FilterResult`. The sealed hierarchy is extensible without breaking existing implementations. -* **No "send to any broker" method.** All requests are addressed by virtual node ID. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. +* **No "send to any broker" method.** All requests are addressed by route and virtual node ID. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. * **Routes generalise filter chains.** A route may carry its own filter list in addition to a target, allowing different filter chains for different paths through the graph. From 0ec38a382db9a9f1655a7795436e8082733fe0a8 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 3 Jun 2026 11:22:41 +1200 Subject: [PATCH 10/30] Add `id` for routes, multi-router topologies Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 49 ++++++++++++++++++++++++++++++++++-- 1 file changed, 47 insertions(+), 2 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 6d3c81f2..9314f856 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -274,12 +274,14 @@ routerDefinitions: config: ... routes: - name: foo + id: 0 filters: - my-first-filter - my-second-filter target: cluster: cluster-a - name: bar + id: 1 filters: - my-third-filter target: @@ -289,6 +291,7 @@ routerDefinitions: config: ... routes: - name: baz + id: 0 target: cluster: cluster-b @@ -311,7 +314,12 @@ The proxy configuration has three top-level definition lists: #### Routes -A route has a `name`, an optional list of `filters` (the names of filters applied to requests/responses traversing this route), and a `target`. +A route has a `name`, an `id`, an optional list of `filters` (the names of filters applied to requests/responses traversing this route), and a `target`. + +The `id` is an integer that identifies the route within the virtual node ID mapping formula (see _Node ID mapping implementation_ below). +Route IDs must be unique within their parent router and in the range `[0, S)` where `S` is the number of routes in the router. +The `id` is separate from the `name` to allow route names to be reordered in the configuration without changing the virtual node ID mapping. +The proxy rejects configurations where the route count `S` is large enough that overflow is inevitable for plausible broker node IDs. The `target` property is a discriminated union containing exactly one of: * `cluster` — the name of a cluster defined in `clusterDefinitions`. @@ -386,6 +394,24 @@ The runtime dispatches incoming requests in one of two modes: * **Dynamic routes**: For all other API keys, the runtime deserialises the request, creates a `RouterContext`, and invokes `Router.onRequest()`. The router uses the context to send requests to routes and deliver a response to the client. +#### Nested router dispatch + +When a router calls `sendRequestToNode()` on a route whose target is another router, the runtime dispatches to the nested router rather than forwarding directly to a backend. +The dispatch works as follows: + +1. The runtime creates (or retrieves from a per-connection cache) a `Router` instance for the nested router. +2. A new `RouterContextImpl` is constructed for the nested router, with its own `NodeIdMapping`, routes, and bootstrap virtual node IDs. + The nested context shares the same client channel, response sequencer, correlation ID allocator, and metrics as the outer context. + Its forwarder callbacks wrap the outer forwarders to translate virtual node IDs from the nested space to the outermost space (see _Per-router scoping and nested dispatch_ above). +3. The runtime invokes `nestedRouter.onRequest()` with the nested context. +4. The `RouterResult` is mapped to a `CompletionStage`: + * `Completed(r)` → the response is returned to the outer router. + * `CompletedNoResponse` → null is returned (fire-and-forget). + * `Disconnect` → the future completes exceptionally (a nested router should not disconnect the client). + +Nested `Router` instances are cached per connection, keyed by the `(outerRoute, routerName)` pair. +They are closed when the client connection closes, in `RouterDispatchHandler.handlerRemoved()`, before the outer router is closed. + #### Correlation ID management When a router sends requests via `RouterContext`, the runtime allocates _routing correlation IDs_ that are distinct from the client's original correlation IDs. @@ -418,7 +444,7 @@ The design supports several mapping strategies, each suited to different deploym **Dedicated mapping (one-to-one).** Each virtual node maps to exactly one `(route, target-broker)` pair. -The current implementation uses a formula: `V = offset + N × t`, where `V` is the virtual node ID, `t` is the target-cluster node ID, `N` is the number of routes, and `offset` is the route's zero-based index. +The current implementation uses a formula: `V = id + S × t`, where `V` is the virtual node ID, `t` is the target-cluster node ID, `S` is the number of routes in the router, and `id` is the route's configured identifier. This mapping is deterministic, stateless, and O(1) in both directions. No coordination between proxy instances is required. @@ -428,10 +454,28 @@ The route parameter in `sendRequestToNode()` is redundant here but serves as a c The dedicated mapping is _stable_: adding or removing brokers from a target cluster does not change the virtual node IDs of existing brokers. Likewise, adding or removing proxy instances has no effect on the mapping. +Because the route's `id` is explicit rather than derived from position, reordering routes in the configuration does not change the mapping. This stability is important because Kafka clients cache broker metadata and will use previously-seen node IDs in subsequent requests. For single-route configurations, an identity mapping (a special case of dedicated mapping) is used: the virtual ID equals the target ID, with zero overhead. +##### Per-router scoping and nested dispatch + +Each router level has its own `NodeIdMapping`, scoped to its routes. +In a nested topology where an outer router has a route targeting an inner router, the inner router works entirely within its own virtual node ID space — its mapping uses `S_inner` (the inner router's route count) and its routes' own `id` values. + +However, the runtime's address resolver maps a single integer virtual node ID to an upstream address. +Nested virtual IDs must therefore be translated to the outermost level before reaching the address resolver. +The translation exploits the outer mapping's unused route slot: for an outer route `r` with `id = k` that targets a nested router, the translation is `V_outer = k + S_outer × V_inner`. +This is simply the outer mapping applied to the nested virtual ID _as if it were a target node ID_. + +Because the outer mapping guarantees uniqueness across its route slots, and the inner mapping guarantees uniqueness within its own space, the composed translation produces globally unique virtual node IDs. +The composition generalises to arbitrary nesting depth: each level translates through its parent route's slot. + +Concretely, this translation is applied by the nested level's forwarder callbacks — the inner `RouterContextImpl` wraps the outer `nodeForwarder` to translate IDs before forwarding to the CCSM. +Address caching from `METADATA` responses also uses the translated IDs, ensuring the CCSM's address resolver can find them. +The `NodeIdResponseTranslator` uses the inner mapping (so the inner router sees its own virtual IDs in responses), while address caching uses the composed translation (so the CCSM sees outer virtual IDs). + **Shared mapping (many-to-one).** Multiple `(route, target-broker)` pairs map to the same virtual node. For example, if proxy instances are presented to clients as broker nodes, each proxy instance handles brokers from multiple routes. @@ -569,6 +613,7 @@ This section summarises the key design choices made in this proposal, for ease o * **No "send to any broker" method.** All requests are addressed by route and virtual node ID. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. +* **Explicit route `id`** decouples the virtual node ID mapping from route ordering. The `id` is an integer in `[0, S)` used in the formula `V = id + S × t`. Making it explicit means reordering routes in the YAML does not change the mapping, avoiding accidental virtual node ID shifts that would invalidate connected clients' cached metadata. * **Routes generalise filter chains.** A route may carry its own filter list in addition to a target, allowing different filter chains for different paths through the graph. * **Route-level SASL** (e.g. SASL initiator for upstream authentication) is achieved via per-route filters rather than a dedicated route-level property. * **Definition names** are global within their type (filters, routers, clusters each have their own namespace) but not across types. From c348beceff45e237bc6428ff5491af51e303e23e Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 3 Jun 2026 11:49:34 +1200 Subject: [PATCH 11/30] Clarify SASL/mTLS, sessionId and Channel Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 9314f856..9273368f 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -219,8 +219,9 @@ Providing a "send to any broker" convenience method would be a footgun for route By requiring a virtual node ID for every request, the router author is forced to think about broker targeting, which correctly reflects the protocol's reality. **`sessionId()`** and **`authenticatedSubject()`** provide observability and identity context. -`authenticatedSubject()` returns the `Subject` established by upstream SASL processing (e.g. a SASL termination filter). -If no SASL processing has occurred, the subject will be anonymous. +`sessionId()` returns a string that uniquely identifies the connection with the Kafka client. It will have the same value for all invocations of `onRequest()` which happen for that client connection, both for the same router over time, and different routers in a topology. +`authenticatedSubject()` returns the client's `Subject` established by mTLS or SASL processing on the VC filter chain (e.g. a SASL termination filter). +If no authentication has occurred, the subject will be anonymous. Correct placement of SASL plugins in the topology is the operator's responsibility; a future proposal may add runtime validation of SASL plugin placement. @@ -401,7 +402,7 @@ The dispatch works as follows: 1. The runtime creates (or retrieves from a per-connection cache) a `Router` instance for the nested router. 2. A new `RouterContextImpl` is constructed for the nested router, with its own `NodeIdMapping`, routes, and bootstrap virtual node IDs. - The nested context shares the same client channel, response sequencer, correlation ID allocator, and metrics as the outer context. + The nested context shares the same Netty client channel, response sequencer, correlation ID allocator, and metrics as the outer context. Its forwarder callbacks wrap the outer forwarders to translate virtual node IDs from the nested space to the outermost space (see _Per-router scoping and nested dispatch_ above). 3. The runtime invokes `nestedRouter.onRequest()` with the nested context. 4. The `RouterResult` is mapped to a `CompletionStage`: @@ -533,7 +534,7 @@ If a router blocks indefinitely (e.g. a backend never responds), the client conn Routers are responsible for implementing their own timeouts if needed. Requests arriving before backend connections are ready are buffered and flushed when the connection becomes active. -Backpressure from any upstream connection propagates to the client channel. +Backpressure from any upstream connection propagates to the Netty client channel. ### Metrics From aebcfea44ca3a1f75a9375cd40c91ea3d93c51cd Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 3 Jun 2026 11:54:37 +1200 Subject: [PATCH 12/30] Clarify errors Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 9273368f..4e52838f 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -255,6 +255,8 @@ This follows the pattern established by `FilterResult`, where the outcome is a v The sealed hierarchy can be extended with new variants (e.g. `CompletedThenDisconnect(Response)`) in future without breaking existing implementations. If the `CompletionStage` completes exceptionally, the runtime treats this as an unrecoverable error and closes the client connection. +In other words, Routers should generally handle errors with the terms of the Kafka protocol, by returning a response to the client which uses appropriate Kafka error codes. +Throwing an exception or returning an exceptionally-completed completion stage is to be avoided where possible. ### Configuration @@ -488,7 +490,7 @@ The route parameter in `sendRequestToNode()` is essential here: without it, the A shared mapping requires the proxy instances to agree on their assignments. If assignments change dynamically (e.g. a proxy instance leaves or joins), a router may hold stale metadata — its cached leader for a topic might reference a virtual node that no longer serves that route. This is not a programming error; it is a natural consequence of eventual consistency, analogous to Kafka's `NOT_LEADER_OR_FOLLOWER`. -The runtime should respond with an appropriate error, triggering the router to refresh its metadata. +The runtime should respond with an appropriate Kafka error code, triggering the router to refresh its metadata. A shared mapping would require strong consistency across proxy instances to ensure they agree on the virtual node ID assignments. From a7fdac97f4f0be63f82de77e8be44ba0debd85c4 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 3 Jun 2026 12:35:08 +1200 Subject: [PATCH 13/30] Tweak wording around deserialization Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 4e52838f..922be864 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -134,8 +134,8 @@ State shared across connections belongs in the `RouterFactory`'s initialisation The router inspects the request, decides which route(s) to use, sends one or more requests via the `RouterContext`, and eventually delivers a response to the client. The returned `CompletionStage` completes when the router has finished processing the request. -**`staticRoutes()`** returns a map of `ApiKeys` to route names for requests that should always be forwarded to a fixed route without deserialisation. -This is a performance optimisation: the runtime can forward these requests as opaque frames, bypassing the cost of deserialisation (assuming no `Filters` require deserialization) and calling `onRequest()`. +**`staticRoutes()`** returns a map of `ApiKeys` to route names for requests that should always be forwarded to a fixed route without `onRequest()` being called. +This allows a potential performance optimisation: if no filters express an interest either the runtime can forward these requests as opaque frames, bypassing the cost of deserialisation. API keys not present in the map are dynamically routed via `onRequest()`. The default implementation returns an empty map (all requests dynamically routed). From ab34f10f5118c0e4f00da3cba908ba67f07a1254 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 8 Jun 2026 10:43:26 +1200 Subject: [PATCH 14/30] Tweaks for some of Rob's points Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 922be864..453649ae 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -25,7 +25,7 @@ For databases, this is known as Data Virtualization. * **Principal-aware routing**. A natural variation on basic [SASL termination](004-terminology-for-authentication.md) is to use the identity of the authenticated client to drive the decision about which backing cluster to route requests to. -This could be used with some metadata about principals to ensure that a client is routed to a cluster that is local to it. +For example, this could be used with some metadata about principals to ensure that a client is routed to a cluster that is local to it. * **Topic splicing**. Multiple separate topics in distinct backing clusters are presented to clients as a single topic. @@ -54,7 +54,7 @@ individual `Router` implementations are free to support as much or as little of To enable the use cases above we need a few concepts: * A _cluster_ is an upstream Kafka cluster that can handle requests. -* A _router_ is a thing which decides which _route(s)_ should be used for a given request. +* A _router_ is a thing which decides which _route(s)_ should be used for a given request and formulates the response to return to the client. * A _route_ is a named pathway from a router towards a _target_. * A route's _target_ is either a cluster or another router. Routes may also have filters attached. From a91b1c9b7b1859eb7317adc387feb54bac4b4c5f Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 8 Jun 2026 11:09:59 +1200 Subject: [PATCH 15/30] More tweaks to address review comments Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 453649ae..a022afc1 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -126,7 +126,7 @@ interface Router { } ``` -A `Router` instance is created **per client connection**, not per virtual cluster. +A `Router` instance (and thus a graph instance) is created **per client connection**, not per virtual cluster. This means per-connection state (caches, session state) can live directly in the `Router` instance without synchronisation, since all calls to a given instance happen on a single Netty event loop thread. State shared across connections belongs in the `RouterFactory`'s initialisation data (see below). @@ -194,7 +194,7 @@ We want to allow (but not require) a router to potentially make multiple request For this reason `RouterContext` does not follow the builder pattern used in the `FilterContext`, but simply exposes methods to asynchronously send requests. This allows the `Router` author to make use of the `CompletionStage` API when issuing multiple requests. -**`bootstrapNodeId(route)`** returns the virtual node ID of a broker on the named route's cluster. +**`bootstrapNodeId(route)`** returns a virtual node ID of a broker on the named route's cluster. This is used to send the initial requests (e.g. `METADATA`) before the router has discovered the cluster's full broker topology. Once `METADATA` responses arrive, the router uses the virtual node IDs from those responses to address specific brokers — no translation is needed, since the node IDs in responses are already virtual (see _Virtual node IDs_ above). From 9063495b2eb8de1bdfa9915492568b3158d3de81 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 12 Jun 2026 05:48:56 +0000 Subject: [PATCH 16/30] Update routing API proposal to reflect implementation changes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sync the proposal with implementation changes made in commits 4d40312bb and 7a1d9e777: - Replace bootstrapNodeId(route) with virtualNodeId() and anyNodeId(route) to distinguish broker-specific vs bootstrap connections - Remove route parameter from sendRequestToNode() — runtime derives it from the virtual node ID via NodeIdMapping.fromVirtual() - Add routeNames() to RouterFactoryContext for validation - Update NodeIdMapping description to reflect fromVirtual() now returns both route and target node ID - Update API_VERSIONS section to use virtualNodeId() - Add note that current implementation uses dedicated mapping only - Update design choices to reflect new API rationale Co-Authored-By: Claude Sonnet 4.5 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 73 +++++++++++++++++++++--------------- 1 file changed, 43 insertions(+), 30 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index a022afc1..3f75354d 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -168,6 +168,7 @@ Because `Router` instances may be created on different event loop threads, the i **`RouterFactoryContext`** provides context to the factory: * `virtualClusterName()` — the name of the virtual cluster. * `routerName()` — the name of this router within the configuration. +* `routeNames()` — the set of route names declared in the router definition. This allows router factories to validate that route names referenced in their plugin configuration actually exist. * `pluginInstance()` / `pluginImplementationNames()` — access to the plugin registry. #### `RouterContext` @@ -177,12 +178,13 @@ Because `Router` instances may be created on different event loop threads, the i ```java interface RouterContext { - int bootstrapNodeId(String route); + OptionalInt virtualNodeId(); - CompletionStage sendRequestToNode(String route, - int virtualNodeId, - RequestHeaderData header, - ApiMessage request); + int anyNodeId(String route); + + CompletionStage sendRequestToNode(int virtualNodeId, + RequestHeaderData header, + ApiMessage request); String sessionId(); @@ -194,29 +196,33 @@ We want to allow (but not require) a router to potentially make multiple request For this reason `RouterContext` does not follow the builder pattern used in the `FilterContext`, but simply exposes methods to asynchronously send requests. This allows the `Router` author to make use of the `CompletionStage` API when issuing multiple requests. -**`bootstrapNodeId(route)`** returns a virtual node ID of a broker on the named route's cluster. -This is used to send the initial requests (e.g. `METADATA`) before the router has discovered the cluster's full broker topology. -Once `METADATA` responses arrive, the router uses the virtual node IDs from those responses to address specific brokers — no translation is needed, since the node IDs in responses are already virtual (see _Virtual node IDs_ above). +**`virtualNodeId()`** returns the virtual node ID of the broker that the client connected to, or empty if the client connected to a bootstrap address. +When the client connects to a broker-specific endpoint (i.e. an address that corresponds to a particular broker in the cluster topology), this returns that broker's virtual node ID. +The router can use this to send requests — such as `API_VERSIONS` — to the specific broker the client believes it is talking to, rather than an arbitrary broker. +This is important during rolling upgrades where different brokers may run different Kafka versions. + +When the client connected to a bootstrap address, this returns empty, because the proxy does not know which broker the client intended. +In that case the router should use `anyNodeId(String)` to obtain a node ID for sending requests. -The runtime is responsible for selecting which broker to return. +**`anyNodeId(route)`** returns a virtual node ID that, when passed to `sendRequestToNode()`, causes the runtime to send the request to an arbitrary broker on the named route's cluster. +This is used for initial discovery requests (e.g. `METADATA`, `FIND_COORDINATOR`) before the router has learned the cluster topology, and for requests that are not broker-specific. +The runtime is responsible for selecting which broker to use. A route's cluster may be configured with multiple bootstrap servers, and the runtime should randomise selection and round-robin on subsequent calls, mirroring how Kafka clients handle bootstrap addresses. This keeps the selection policy in the runtime, where it can be applied consistently, rather than requiring each router to implement its own strategy. -**`sendRequestToNode(route, virtualNodeId, ...)`** sends a request to a specific broker, identified by the route and a virtual node ID. -The runtime uses the route to determine which target cluster, and resolves the virtual node ID to a specific upstream broker address, opening a new connection if necessary. +**`sendRequestToNode(virtualNodeId, ...)`** sends a request to a specific broker, identified by a virtual node ID. +The runtime derives the route from the virtual node ID (via the `NodeIdMapping`) and resolves it to a specific upstream broker address, opening a new connection if necessary. -The router passes both the route (which it knows from its routing decision) and the virtual node ID (which it knows from METADATA or `bootstrapNodeId()`). -Both pieces of information are naturally available at every call site — the route is the fundamental output of the routing decision, and the virtual node ID comes from cached metadata. +The virtual node ID can be: +* A value obtained from `virtualNodeId()` — sends to the broker the client connected to +* A value obtained from `anyNodeId(String)` — sends to an arbitrary broker on a route +* A virtual node ID learned from a previous response (e.g. a partition leader from a `METADATA` response) -The route parameter is essential for supporting different `NodeIdMapping` strategies (see _Node ID mapping implementation_ below). -With a dedicated (one-to-one) mapping, each virtual node belongs to exactly one route, so the route is redundant but serves as a cross-check. -With a shared (many-to-one) mapping — for example, if proxy instances were presented to clients as broker nodes — a single virtual node may serve brokers from multiple routes, and the route parameter is the only way for the runtime to determine which target cluster the request should reach. -Including the route in all cases avoids locking router implementations into a specific mapping strategy: the same router code works regardless of which mapping the runtime uses. +Once `METADATA` responses arrive, the router uses the virtual node IDs from those responses to address specific brokers — no translation is needed, since the node IDs in responses are already virtual (see _Virtual node IDs_ above). -There is no "send to any broker" method. -This is a deliberate design choice: the Kafka protocol scopes most operations to a specific broker (partition leaders, group coordinators, per-broker `API_VERSIONS` negotiation). -Providing a "send to any broker" convenience method would be a footgun for router authors — it would be easy to use for an API that actually requires a specific broker, and the resulting bug would only manifest at runtime under specific conditions (e.g. multi-broker clusters, rolling upgrades). -By requiring a virtual node ID for every request, the router author is forced to think about broker targeting, which correctly reflects the protocol's reality. +The removal of the route parameter from `sendRequestToNode()` compared to earlier drafts is deliberate: the runtime can derive the route from the virtual node ID via `NodeIdMapping.fromVirtual()`. +This works for dedicated (one-to-one) mappings where each virtual node belongs to exactly one route. +For shared (many-to-one) mappings — where a single virtual node may serve brokers from multiple routes — the mapping implementation would need to maintain additional state to recover the route from the virtual node ID, or router implementations would need to be restricted to dedicated mappings only. **`sessionId()`** and **`authenticatedSubject()`** provide observability and identity context. `sessionId()` returns a string that uniquely identifies the connection with the Kafka client. It will have the same value for all invocations of `onRequest()` which happen for that client connection, both for the same router over time, and different routers in a topology. @@ -435,11 +441,12 @@ As described in _Virtual node IDs_ above, the runtime translates between real ta This is implemented by a `NodeIdMapping` abstraction (internal to the runtime) with two operations: * `toVirtual(route, targetNodeId)` — used when rewriting responses received from a target cluster (the runtime rewrites broker node IDs in `METADATA`, `PRODUCE`, `FIND_COORDINATOR`, and `DESCRIBE_CLUSTER` responses before they reach the router). -* `fromVirtual(route, virtualNodeId)` — used when the router calls `sendRequestToNode(route, virtualNodeId, ...)`, returning the target-cluster node ID. +* `fromVirtual(virtualNodeId)` — used when the router calls `sendRequestToNode(virtualNodeId, ...)`, returning both the route and the target-cluster node ID. -The mapping must satisfy: `fromVirtual(route, toVirtual(route, t))` returns `t` for any route and target node ID `t`. -Note that the mapping does not need to recover the route from the virtual node ID alone — the route is always supplied by the router. -This allows both _dedicated_ (one-to-one) and _shared_ (many-to-one) mapping strategies. +The mapping must satisfy: for any route `r` and target node ID `t`, if `v = toVirtual(r, t)` then `fromVirtual(v)` returns `(r, t)`. +The mapping must be able to recover the route from the virtual node ID alone. +This is straightforward for _dedicated_ (one-to-one) mappings where each virtual node belongs to exactly one route. +For _shared_ (many-to-one) mappings, the mapping implementation would need to maintain additional state to recover the route from the virtual node ID. ##### Mapping strategies @@ -453,7 +460,7 @@ No coordination between proxy instances is required. The client sees all brokers from all clusters (each with a unique virtual ID). Because each virtual node belongs to exactly one route, leader-directed requests (e.g. `PRODUCE`, `FETCH`) from a well-behaved client will only contain topics from one route — no decomposition is needed for these APIs. -The route parameter in `sendRequestToNode()` is redundant here but serves as a cross-check. +The runtime can derive the route from the virtual node ID via `fromVirtual()`. The dedicated mapping is _stable_: adding or removing brokers from a target cluster does not change the virtual node IDs of existing brokers. Likewise, adding or removing proxy instances has no effect on the mapping. @@ -485,7 +492,7 @@ For example, if proxy instances are presented to clients as broker nodes, each p The client sees fewer nodes (one per proxy instance rather than one per target broker). Because a single virtual node may serve brokers from multiple routes, a leader-directed request to one virtual node _can_ contain topics from different routes — decomposition is needed even for `PRODUCE` and `FETCH`. -The route parameter in `sendRequestToNode()` is essential here: without it, the runtime cannot determine which target cluster the request should reach. +For the runtime to support this, `fromVirtual()` would need to maintain additional state to recover which route was used when the virtual node ID was created, or the mapping would need to be restricted such that each virtual node still maps to exactly one route (making it effectively a dedicated mapping). A shared mapping requires the proxy instances to agree on their assignments. If assignments change dynamically (e.g. a proxy instance leaves or joins), a router may hold stale metadata — its cached leader for a topic might reference a virtual node that no longer serves that route. @@ -494,6 +501,8 @@ The runtime should respond with an appropriate Kafka error code, triggering the A shared mapping would require strong consistency across proxy instances to ensure they agree on the virtual node ID assignments. +Note: The current implementation uses a dedicated mapping and does not support shared mappings. Supporting shared mappings would require changes to the `NodeIdMapping` interface to allow `fromVirtual()` to recover the route. + **Role-based mapping.** Virtual nodes correspond to per-route functional roles (e.g. "partition leader", "group coordinator"). If roles are scoped per route, this behaves like a dedicated mapping. @@ -512,7 +521,9 @@ Changing the mapping strategy (e.g. from dedicated to shared) is expected to be The Kafka protocol scopes `API_VERSIONS` to a single broker connection — a client sends `API_VERSIONS` each time it connects to a new broker. This means the proxy must forward `API_VERSIONS` to the specific broker the client believes it is talking to, not to an arbitrary broker. -With a dedicated mapping, each virtual node ID maps to exactly one real broker, so the router can use `sendRequestToNode()` to forward `API_VERSIONS` to the correct broker. +When the client connects to a broker-specific endpoint, `virtualNodeId()` returns that broker's virtual node ID, allowing the router to forward `API_VERSIONS` to the correct broker via `sendRequestToNode()`. +When the client connects to a bootstrap address, `virtualNodeId()` returns empty, and the router should use `anyNodeId(route)` to send `API_VERSIONS` to an arbitrary broker on the route. + With a shared mapping, the runtime would need to fan out `API_VERSIONS` to all brokers behind that virtual node and return the intersection of their version ranges. The runtime's existing `ApiVersionsIntersectFilter` intersects each backend broker's version ranges with the proxy's own maximums. @@ -610,10 +621,12 @@ Having a single `clusterDefinitions` list as the authoritative catalogue of upst This section summarises the key design choices made in this proposal, for ease of reference. * **Virtual node IDs** provide a uniform addressing scheme that insulates router authors from the node ID collision problem inherent in multi-cluster topologies. The runtime owns the mapping; routers work exclusively with virtual IDs and never need to translate. -* **Route included in `sendRequestToNode`** to support both dedicated (one-to-one) and shared (many-to-one) node ID mappings. With a dedicated mapping the route is redundant; with a shared mapping it is essential. Including it in all cases avoids locking router implementations into a specific mapping strategy, allowing the same router code to work across deployment topologies (e.g. each-broker-is-a-node vs proxy-instances-as-nodes). +* **Separate `virtualNodeId()` and `anyNodeId(route)` methods** distinguish between broker-specific connections (where the client connected to a specific broker endpoint) and bootstrap connections (where the client connected to a generic bootstrap address). This allows routers to correctly handle `API_VERSIONS` requests by forwarding them to the specific broker the client connected to, which is important during rolling upgrades where different brokers may run different Kafka versions. +* **Route derived from virtual node ID** — `sendRequestToNode()` takes only a virtual node ID, and the runtime derives the route via `NodeIdMapping.fromVirtual()`. This works for dedicated (one-to-one) mappings where each virtual node belongs to exactly one route. Shared (many-to-one) mappings would require the mapping to maintain additional state to recover the route, or be restricted such that each virtual node still maps to exactly one route. * **Per-connection `Router` instances** allow per-connection state (caches, sessions) without synchronisation. Shared state lives in the `RouterFactory`'s initialisation data, which must be thread-safe. * **`RouterResult` as a sealed type** encodes request outcomes (response, no-response, disconnect) as values rather than side-effects, following the pattern established by `FilterResult`. The sealed hierarchy is extensible without breaking existing implementations. -* **No "send to any broker" method.** All requests are addressed by route and virtual node ID. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. +* **All requests addressed by virtual node ID.** Routers must explicitly obtain a virtual node ID (via `virtualNodeId()`, `anyNodeId(route)`, or from a previous response) before sending a request. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. +* **`routeNames()` in `RouterFactoryContext`** allows router factories to validate at initialization time that route names referenced in their plugin configuration actually exist, providing early feedback on configuration errors. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. * **Explicit route `id`** decouples the virtual node ID mapping from route ordering. The `id` is an integer in `[0, S)` used in the formula `V = id + S × t`. Making it explicit means reordering routes in the YAML does not change the mapping, avoiding accidental virtual node ID shifts that would invalidate connected clients' cached metadata. From a414279d6e00ef58a67232d2bb2618334f64dd65 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Fri, 12 Jun 2026 05:58:11 +0000 Subject: [PATCH 17/30] Update routing API proposal for builder pattern and topicName() Sync the proposal with implementation changes made since commit 5b6545d72: - Replace RouterResult sealed type with RouterResponse and builder pattern - Add respondWith(), respondWithError(), respondWithoutReply() builder methods to RouterContext following the Filter API pattern - Add CloseOrTerminalStage, TerminalStage, CloseStage interfaces for fluent response construction with optional connection closure - Add topicName(Uuid) method to RouterContext for synchronous topic ID resolution (necessary for APIs like SHARE_FETCH that lack topic names) - Update all RouterResult references to RouterResponse - Update Response interface references (now returns ApiMessage directly) - Update design choices section to reflect builder pattern and topicName() The builder pattern allows routers to construct responses via context.respondWith(body).build() instead of new RouterResult.Completed(), consistent with how filters work and allowing the API to evolve without breaking changes. Co-Authored-By: Claude Sonnet 4.5 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 116 +++++++++++++++++++++++++---------- 1 file changed, 83 insertions(+), 33 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 3f75354d..4c4777c3 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -112,11 +112,11 @@ In contrast, a `Router` implementing the 'topic splicing' use case might require ```java interface Router { - CompletionStage onRequest(short apiVersion, - ApiKeys apiKey, - RequestHeaderData header, - ApiMessage request, - RouterContext context); + CompletionStage onRequest(short apiVersion, + ApiKeys apiKey, + RequestHeaderData header, + ApiMessage request, + RouterContext context); default Map staticRoutes() { return Map.of(); @@ -131,8 +131,8 @@ This means per-connection state (caches, session state) can live directly in the State shared across connections belongs in the `RouterFactory`'s initialisation data (see below). **`onRequest()`** is invoked for each incoming client request that is _dynamically routed_. -The router inspects the request, decides which route(s) to use, sends one or more requests via the `RouterContext`, and eventually delivers a response to the client. -The returned `CompletionStage` completes when the router has finished processing the request. +The router inspects the request, decides which route(s) to use, sends one or more requests via the `RouterContext`, and eventually delivers a response to the client using the builder pattern. +The returned `CompletionStage` completes when the router has finished processing the request. **`staticRoutes()`** returns a map of `ApiKeys` to route names for requests that should always be forwarded to a fixed route without `onRequest()` being called. This allows a potential performance optimisation: if no filters express an interest either the runtime can forward these requests as opaque frames, bypassing the cost of deserialisation. @@ -189,6 +189,18 @@ interface RouterContext { String sessionId(); Subject authenticatedSubject(); + + String topicName(Uuid topicId); + + CloseOrTerminalStage respondWith(ApiMessage body); + + CloseOrTerminalStage respondWith(ResponseHeaderData header, ApiMessage body); + + CloseOrTerminalStage respondWithError(RequestHeaderData header, + ApiMessage request, + ApiException exception); + + CloseOrTerminalStage respondWithoutReply(); } ``` @@ -231,38 +243,75 @@ If no authentication has occurred, the subject will be anonymous. Correct placement of SASL plugins in the topology is the operator's responsibility; a future proposal may add runtime validation of SASL plugin placement. +**`topicName(topicId)`** resolves a topic ID to its topic name synchronously. +The runtime guarantees that all topic IDs present in the current request have been resolved before `Router.onRequest()` is called, so this method returns immediately from a per-connection cache. +The cache is populated by an internal filter that sends `METADATA` requests on cache miss. +This is necessary for Kafka APIs (such as `SHARE_FETCH`) that have only a topic ID field with no topic name property. +Returns `null` if the topic ID could not be resolved (e.g. the topic was deleted). + +**Response builder methods** — `respondWith()`, `respondWithError()`, and `respondWithoutReply()` — follow a fluent builder pattern consistent with the `Filter` API. +Rather than constructing `RouterResult` subtypes directly, routers use the builder to construct responses: +* `context.respondWith(body).build()` — delivers a response to the client +* `context.respondWith(header, body).build()` — delivers a synthesised response with a custom header +* `context.respondWithError(header, request, exception).build()` — generates an API-specific error response +* `context.respondWithoutReply().build()` — completes with no response (fire-and-forget) + +Each builder method returns a `CloseOrTerminalStage`, which supports optional connection closure via `.andCloseConnection()` before calling `.build()`: +* `context.respondWith(body).andCloseConnection().build()` — delivers a response then closes the connection + All `RouterContext` methods are called on the same Netty event loop thread as `onRequest()`. No synchronisation is needed within a single `Router` instance. -#### `RouterResult` +#### `RouterResponse` and builder pattern -`RouterResult` is the return type from `onRequest()`. -It is a sealed type that encodes the outcome of request processing: +`RouterResponse` is the return type from `onRequest()`. +It is an opaque interface constructed via the builder pattern on `RouterContext`: ```java -sealed interface RouterResult { - record Completed(Response response) implements RouterResult {} - record CompletedNoResponse() implements RouterResult {} - record Disconnect() implements RouterResult {} +interface RouterResponse {} + +interface CloseOrTerminalStage { + TerminalStage andCloseConnection(); + RouterResponse build(); +} + +interface TerminalStage { + RouterResponse build(); } ``` -**`Completed(response)`** — the router has produced a response for the client. -The runtime delivers the response, automatically rewriting the correlation ID to match the client's original request. +Routers construct responses using the builder methods on `RouterContext`: + +```java +// Deliver a response to the client +return context.respondWith(responseBody).build(); + +// Deliver a response with a custom header +return context.respondWith(responseHeader, responseBody).build(); + +// Generate an error response +return context.respondWithError(requestHeader, request, exception).build(); -**`CompletedNoResponse()`** — the router has finished processing but there is no response to deliver. -This is the correct result for fire-and-forget requests (e.g. `PRODUCE` with `acks=0`). -Having a dedicated type rather than a nullable response field makes the `acks=0` case explicit. -The runtime can log a warning if a router returns `Completed` for a request that has no response, or `CompletedNoResponse` for a request that expects one. +// Complete without sending a response (fire-and-forget) +return context.respondWithoutReply().build(); -**`Disconnect()`** — the router requests that the client connection be closed. +// Deliver a response and close the connection +return context.respondWith(responseBody).andCloseConnection().build(); +``` + +The builder pattern is consistent with the `Filter` API, where outcomes are constructed via a fluent interface rather than by directly instantiating result types. +This allows the API to evolve (e.g. adding new builder stages for metrics or tracing) without breaking existing router implementations. + +**Response delivery.** When a router returns a response via `respondWith()`, the runtime automatically rewrites the correlation ID to match the client's original request. +Routers never need to manage correlation IDs themselves. -This follows the pattern established by `FilterResult`, where the outcome is a value returned to the runtime rather than an imperative side-effect on the context. -The sealed hierarchy can be extended with new variants (e.g. `CompletedThenDisconnect(Response)`) in future without breaking existing implementations. +**Fire-and-forget.** For requests that expect no response (e.g. `PRODUCE` with `acks=0`), routers use `respondWithoutReply()`. +Having a dedicated builder method rather than a nullable response makes the fire-and-forget case explicit. +The runtime can log a warning if a router uses `respondWith()` for a request that has no response, or `respondWithoutReply()` for a request that expects one. -If the `CompletionStage` completes exceptionally, the runtime treats this as an unrecoverable error and closes the client connection. -In other words, Routers should generally handle errors with the terms of the Kafka protocol, by returning a response to the client which uses appropriate Kafka error codes. -Throwing an exception or returning an exceptionally-completed completion stage is to be avoided where possible. +**Error handling.** If the `CompletionStage` completes exceptionally, the runtime treats this as an unrecoverable error and closes the client connection. +Routers should generally handle errors within the terms of the Kafka protocol, using `respondWithError()` to generate error responses with appropriate Kafka error codes. +Throwing an exception or returning an exceptionally-completed stage should be avoided where possible. ### Configuration @@ -413,10 +462,10 @@ The dispatch works as follows: The nested context shares the same Netty client channel, response sequencer, correlation ID allocator, and metrics as the outer context. Its forwarder callbacks wrap the outer forwarders to translate virtual node IDs from the nested space to the outermost space (see _Per-router scoping and nested dispatch_ above). 3. The runtime invokes `nestedRouter.onRequest()` with the nested context. -4. The `RouterResult` is mapped to a `CompletionStage`: - * `Completed(r)` → the response is returned to the outer router. - * `CompletedNoResponse` → null is returned (fire-and-forget). - * `Disconnect` → the future completes exceptionally (a nested router should not disconnect the client). +4. The `RouterResponse` is unwrapped to a `CompletionStage`: + * Response built via `respondWith()` → the response body is returned to the outer router. + * Response built via `respondWithoutReply()` → null is returned (fire-and-forget). + * Response built via `andCloseConnection()` → the future completes exceptionally (a nested router should not disconnect the client). Nested `Router` instances are cached per connection, keyed by the `(outerRoute, routerName)` pair. They are closed when the client connection closes, in `RouterDispatchHandler.handlerRemoved()`, before the outer router is closed. @@ -535,7 +584,7 @@ Version capping is the router's responsibility, not the runtime's. #### Flow control The runtime uses Netty's auto-read mechanism to apply TCP-level backpressure to the client connection. -Auto-read is disabled while the router is processing a request and re-enabled when the `CompletionStage` completes. +Auto-read is disabled while the router is processing a request and re-enabled when the `CompletionStage` completes. This means the router's processing strategy directly determines the proxy's response latency and backpressure behaviour: * A router that forwards a request to a single backend and returns the response has latency equal to the backend's latency. @@ -584,7 +633,7 @@ This cardinality is comparable to the existing per-filter metrics and should not ## Affected/not affected projects -* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResult`, `Response`. +* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResponse`, `CloseOrTerminalStage`, `TerminalStage`, `CloseStage`. * `kroxylicious-runtime` — routing engine, configuration model (`RouterDefinition`, `RouteDefinition`, `TargetClusterDefinition`), graph validation, node ID mapping, response sequencing, metrics. * `kroxylicious-bom` — version management for any new modules. * Not affected: existing filters, KMS, authoriser API. The Kubernetes operator will need a separate update to support router configuration in CRDs. @@ -624,8 +673,9 @@ This section summarises the key design choices made in this proposal, for ease o * **Separate `virtualNodeId()` and `anyNodeId(route)` methods** distinguish between broker-specific connections (where the client connected to a specific broker endpoint) and bootstrap connections (where the client connected to a generic bootstrap address). This allows routers to correctly handle `API_VERSIONS` requests by forwarding them to the specific broker the client connected to, which is important during rolling upgrades where different brokers may run different Kafka versions. * **Route derived from virtual node ID** — `sendRequestToNode()` takes only a virtual node ID, and the runtime derives the route via `NodeIdMapping.fromVirtual()`. This works for dedicated (one-to-one) mappings where each virtual node belongs to exactly one route. Shared (many-to-one) mappings would require the mapping to maintain additional state to recover the route, or be restricted such that each virtual node still maps to exactly one route. * **Per-connection `Router` instances** allow per-connection state (caches, sessions) without synchronisation. Shared state lives in the `RouterFactory`'s initialisation data, which must be thread-safe. -* **`RouterResult` as a sealed type** encodes request outcomes (response, no-response, disconnect) as values rather than side-effects, following the pattern established by `FilterResult`. The sealed hierarchy is extensible without breaking existing implementations. +* **Builder pattern for response construction** follows the same fluent interface pattern as `FilterResult`, where outcomes are constructed via `RouterContext` builder methods rather than by directly instantiating result types. This allows the API to evolve (e.g. adding new builder stages) without breaking existing implementations. * **All requests addressed by virtual node ID.** Routers must explicitly obtain a virtual node ID (via `virtualNodeId()`, `anyNodeId(route)`, or from a previous response) before sending a request. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. +* **`topicName(Uuid)` for synchronous topic ID resolution** allows routers to resolve topic IDs to names without relying on wire-object enrichment. This is necessary for Kafka APIs (such as `SHARE_FETCH`) that have only a topic ID field. The runtime guarantees the cache is warm before `onRequest()` is called. * **`routeNames()` in `RouterFactoryContext`** allows router factories to validate at initialization time that route names referenced in their plugin configuration actually exist, providing early feedback on configuration errors. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. From f80732a904cadc8f800f208cc8ccc506fb394246 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 15 Jun 2026 21:17:12 +0000 Subject: [PATCH 18/30] docs: update routing API for VirtualNode and TopologyService Updates the proposal to reflect the implemented API: - VirtualNode replaces int virtual node IDs as an opaque reference type, enabling alternative networking models (proxy-as-broker) - nodeForId(int) bridges the Kafka wire protocol to VirtualNode - RouterContext methods renamed: virtualNode(), anyNode(), sendRequest() - TopologyService: opt-in topology cache with side-effect population from METADATA responses, coarse invalidation, router-driven staleness handling - PartitionInfo and BrokerInfo for follower-fetch / AZ-aware routing - Shared node address map (per-router-level, not per-connection) - RequestSender binding for TopologyService async methods Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 212 +++++++++++++++++++++++++++-------- 1 file changed, 168 insertions(+), 44 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 4c4777c3..8f546ee0 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -76,26 +76,35 @@ The _router_ decides which _route_ each request should traverse. Each _route_ may carry its own filter chain, and terminates at a _cluster_ (or another router). Clusters are the leaf nodes of the graph. -#### Virtual node IDs +#### VirtualNode Kafka's protocol identifies brokers by integer _node IDs_. These node IDs are scoped to a single cluster — broker 0 in cluster-a is a completely different machine from broker 0 in cluster-b. When a router presents multiple clusters to the client as a single virtual cluster, there is a collision problem: the client might receive node ID 0 in a `METADATA` response from one route and node ID 0 from another, with no way to distinguish them. -The solution is _virtual node IDs_. -The runtime assigns each `(route, target-cluster node ID)` pair a unique virtual node ID. -Virtual node IDs are `int32`, matching Kafka's wire format for node IDs. We believe this is large enough not to be a practical limit. -All responses delivered to the router (and onward to the client) use virtual node IDs. -All requests from the router use virtual node IDs. -The runtime transparently translates between virtual and real node IDs at the boundary. +The solution is `VirtualNode` — an opaque reference type for node identity in the routing API. +The runtime assigns each `(route, target-cluster node ID)` pair a unique `VirtualNode`. +Routers obtain `VirtualNode` instances from `RouterContext` methods and pass them back to `sendRequest()`. +The type is intentionally opaque — routers must not inspect or construct instances; the runtime provides the implementation, which includes correct `equals()` and `hashCode()` so that `VirtualNode` instances can be used as map keys. -This means a router author works entirely in terms of virtual node IDs and never needs to translate. -When a `METADATA` response arrives with a list of brokers, the node IDs in that response are already virtual. -When the router wants to send a request to one of those brokers, it simply passes the virtual node ID back. -The runtime handles the mapping. +All responses delivered to the router (and onward to the client) use integer node IDs on the Kafka wire protocol. +The router converts these to `VirtualNode` via `nodeForId(int)` when it needs to use them as routing handles. +All requests from the router use `VirtualNode` via `sendRequest()`. +The runtime transparently translates between `VirtualNode` and real node IDs at the boundary. -The details of how the mapping works (the formula, the implementation) are described in the _Runtime_ section below. -The key point here is that virtual node IDs are the universal addressing scheme within the routing API. +**Why opaque, not `int`?** +The integer-based port-per-broker networking model encodes both route and target broker into a single `int` via a bijective formula. +An alternative networking model — where proxy instances act as brokers with their own identities and shard-based ownership — needs richer routing information that cannot be encoded in a single integer. +`VirtualNode` hides this difference: the same router code works with either networking model. +The runtime implementation (`VirtualNodeImpl`) wraps the encoded integer for the current model; a future implementation could carry proxy instance identity, shard keys, or other routing metadata. + +**`nodeForId(int)` — the wire-protocol bridge.** +Kafka protocol messages (METADATA responses, FIND_COORDINATOR responses, etc.) carry node IDs as integers. +When a router reads these integers — for example, broker node IDs in a METADATA response during fan-out merge, or a coordinator node ID from FIND_COORDINATOR — it converts them to `VirtualNode` via `nodeForId(int)`. +This method is permanent, not transitional: routers will always need to interpret integers from protocol messages. + +The details of how the internal mapping works (the formula, the implementation) are described in the _Runtime_ section below. +The key point here is that `VirtualNode` is the universal addressing scheme within the routing API, and `nodeForId(int)` is the bridge from the Kafka wire protocol to that scheme. ### Plugin API @@ -170,6 +179,7 @@ Because `Router` instances may be created on different event loop threads, the i * `routerName()` — the name of this router within the configuration. * `routeNames()` — the set of route names declared in the router definition. This allows router factories to validate that route names referenced in their plugin configuration actually exist. * `pluginInstance()` / `pluginImplementationNames()` — access to the plugin registry. +* `topologyService()` — returns a `TopologyService` for opt-in topology caching (see _TopologyService_ below). The runtime creates the underlying cache lazily on first call. Routers that never call this method pay no cost. The returned service should be stored in the factory's initialisation data so it survives connection reconnects. #### `RouterContext` @@ -178,13 +188,15 @@ Because `Router` instances may be created on different event loop threads, the i ```java interface RouterContext { - OptionalInt virtualNodeId(); + Optional virtualNode(); + + VirtualNode anyNode(String route); - int anyNodeId(String route); + VirtualNode nodeForId(int virtualNodeId); - CompletionStage sendRequestToNode(int virtualNodeId, - RequestHeaderData header, - ApiMessage request); + CompletionStage sendRequest(VirtualNode node, + RequestHeaderData header, + ApiMessage request); String sessionId(); @@ -208,33 +220,37 @@ We want to allow (but not require) a router to potentially make multiple request For this reason `RouterContext` does not follow the builder pattern used in the `FilterContext`, but simply exposes methods to asynchronously send requests. This allows the `Router` author to make use of the `CompletionStage` API when issuing multiple requests. -**`virtualNodeId()`** returns the virtual node ID of the broker that the client connected to, or empty if the client connected to a bootstrap address. -When the client connects to a broker-specific endpoint (i.e. an address that corresponds to a particular broker in the cluster topology), this returns that broker's virtual node ID. +**`virtualNode()`** returns the `VirtualNode` of the broker that the client connected to, or empty if the client connected to a bootstrap address. +When the client connects to a broker-specific endpoint (i.e. an address that corresponds to a particular broker in the cluster topology), this returns that broker's `VirtualNode`. The router can use this to send requests — such as `API_VERSIONS` — to the specific broker the client believes it is talking to, rather than an arbitrary broker. This is important during rolling upgrades where different brokers may run different Kafka versions. When the client connected to a bootstrap address, this returns empty, because the proxy does not know which broker the client intended. -In that case the router should use `anyNodeId(String)` to obtain a node ID for sending requests. +In that case the router should use `anyNode(String)` to obtain a node for sending requests. -**`anyNodeId(route)`** returns a virtual node ID that, when passed to `sendRequestToNode()`, causes the runtime to send the request to an arbitrary broker on the named route's cluster. +**`anyNode(route)`** returns a `VirtualNode` that, when passed to `sendRequest()`, causes the runtime to send the request to an arbitrary broker on the named route's cluster. This is used for initial discovery requests (e.g. `METADATA`, `FIND_COORDINATOR`) before the router has learned the cluster topology, and for requests that are not broker-specific. The runtime is responsible for selecting which broker to use. A route's cluster may be configured with multiple bootstrap servers, and the runtime should randomise selection and round-robin on subsequent calls, mirroring how Kafka clients handle bootstrap addresses. This keeps the selection policy in the runtime, where it can be applied consistently, rather than requiring each router to implement its own strategy. -**`sendRequestToNode(virtualNodeId, ...)`** sends a request to a specific broker, identified by a virtual node ID. -The runtime derives the route from the virtual node ID (via the `NodeIdMapping`) and resolves it to a specific upstream broker address, opening a new connection if necessary. +**`nodeForId(int)`** converts an integer node ID from a Kafka protocol response body into a `VirtualNode`. +This is the bridge between the Kafka wire protocol (which uses integer node IDs) and the opaque `VirtualNode` API. +Routers need this when interpreting node IDs in protocol messages — for example, broker node IDs when merging `METADATA` responses from multiple routes, coordinator node IDs from `FIND_COORDINATOR` responses, or leader IDs in partition metadata. -The virtual node ID can be: -* A value obtained from `virtualNodeId()` — sends to the broker the client connected to -* A value obtained from `anyNodeId(String)` — sends to an arbitrary broker on a route -* A virtual node ID learned from a previous response (e.g. a partition leader from a `METADATA` response) +**`sendRequest(node, ...)`** sends a request to a specific broker, identified by a `VirtualNode`. +The runtime derives the route from the `VirtualNode` (via the internal `NodeIdMapping`) and resolves it to a specific upstream broker address, opening a new connection if necessary. -Once `METADATA` responses arrive, the router uses the virtual node IDs from those responses to address specific brokers — no translation is needed, since the node IDs in responses are already virtual (see _Virtual node IDs_ above). +The `VirtualNode` can be: +* A value obtained from `virtualNode()` — sends to the broker the client connected to +* A value obtained from `anyNode(String)` — sends to an arbitrary broker on a route +* A value obtained from `nodeForId(int)` — sends to a broker whose ID was learned from a protocol response -The removal of the route parameter from `sendRequestToNode()` compared to earlier drafts is deliberate: the runtime can derive the route from the virtual node ID via `NodeIdMapping.fromVirtual()`. +Once `METADATA` responses arrive, the router converts the integer node IDs from those responses to `VirtualNode` via `nodeForId(int)`, then uses those nodes to address specific brokers via `sendRequest()`. + +The absence of a route parameter on `sendRequest()` is deliberate: the runtime can derive the route from the `VirtualNode` via the internal `NodeIdMapping.fromVirtual()`. This works for dedicated (one-to-one) mappings where each virtual node belongs to exactly one route. -For shared (many-to-one) mappings — where a single virtual node may serve brokers from multiple routes — the mapping implementation would need to maintain additional state to recover the route from the virtual node ID, or router implementations would need to be restricted to dedicated mappings only. +For shared (many-to-one) mappings — where a single virtual node may serve brokers from multiple routes — the `VirtualNode` implementation can encode the necessary routing metadata (proxy instance identity, shard assignments, etc.) without changing the `sendRequest()` API. **`sessionId()`** and **`authenticatedSubject()`** provide observability and identity context. `sessionId()` returns a string that uniquely identifies the connection with the Kafka client. It will have the same value for all invocations of `onRequest()` which happen for that client connection, both for the same router over time, and different routers in a topology. @@ -313,6 +329,77 @@ The runtime can log a warning if a router uses `respondWith()` for a request tha Routers should generally handle errors within the terms of the Kafka protocol, using `respondWithError()` to generate error responses with appropriate Kafka error codes. Throwing an exception or returning an exceptionally-completed stage should be avoided where possible. +#### `TopologyService` + +`TopologyService` is an opt-in topology cache for routers that need leader, coordinator, broker, or topic ID information. +Routers obtain it from `RouterFactoryContext.topologyService()` during `RouterFactory.initialize()`. +Routers that never call `topologyService()` pay no cost — no cache is created, and the runtime skips topology-related work for that router level. + +```java +interface TopologyService { + CompletionStage> topicNames(Set topicIds); + + Optional leaderOf(String topicName, int partitionIndex); + CompletionStage ensureLeadersCached(Map> topicsByRoute); + + Optional coordinatorOf(String route, byte keyType, String key); + CompletionStage discoverCoordinator(String route, byte keyType, String key); + + Optional partitionInfoFor(String topicName, int partitionIndex); + Optional brokerInfo(VirtualNode node); + + void invalidateRoute(String route); +} +``` + +The service provides three categories of methods: + +**Synchronous reads** — `leaderOf()`, `coordinatorOf()`, `partitionInfoFor()`, `brokerInfo()` — query the cache and return immediately. +If the data is not cached, they return empty. +Callers should warm the cache first (via `ensureLeadersCached()`, `discoverCoordinator()`, or by sending METADATA directly). + +**Asynchronous operations** — `ensureLeadersCached()`, `discoverCoordinator()`, `topicNames()` — may send requests internally to warm the cache. +`ensureLeadersCached()` batches all uncached topics into one METADATA request per route. +`discoverCoordinator()` sends METADATA (if needed) then FIND_COORDINATOR. +`topicNames()` resolves topic IDs to names, batching cache misses into a single METADATA request. +These methods use a `RequestSender` bound per-connection by the runtime (see _Runtime_ below). + +**Invalidation** — `invalidateRoute(route)` performs coarse invalidation: clears all partition info (leaders, replicas, ISR), coordinators, and broker info for a route. +Topic ID→name mappings are _not_ cleared (they are stable within a cluster — a topic ID always maps to the same name). +Over-invalidation is acceptable because the cache is repopulated cheaply from the client's next METADATA request. + +**Cache population model.** +The cache is populated as a _side effect_ of METADATA responses flowing through the routing pipeline. +When any request sent via `RouterContext.sendRequest()` produces a METADATA response, the runtime intercepts the response in `RoutingDecisionHandler.write()` (after node ID translation) and updates the topology cache _before_ completing the router's `CompletionStage`. +This means: after a METADATA request's future completes, the cache is guaranteed to reflect that response. +Routers that need to warm the cache for uncached topics send METADATA requests themselves (or via `ensureLeadersCached()`), and the cache is populated as a side effect of the response flow. + +Coordinator caching works differently: the FIND_COORDINATOR response lacks `keyType`, and pre-v4 responses lack the `key` field entirely. +So coordinators cannot be cached from the response alone. +The `TopologyServiceImpl.discoverCoordinator()` method caches them explicitly using request-side context (the key type and key from the original request). +Coordinators are keyed by `(route, keyType, key)` — not by route alone, fixing a limitation where different consumer groups on the same route could collide. + +Only METADATA responses populate partition and broker data. +DESCRIBE_CLUSTER is intentionally excluded: it contains brokers and racks but no topic/partition/leader information, and mixing data from different response types risks inconsistency. + +**Cache scope.** +The cache is shared per router level (not per connection), backed by `ConcurrentHashMap`. +All connections through the same router level share the same topology view. +This is efficient (no per-connection duplication) and necessary for correctness: a leader discovered by connection A must be usable by connection B (see _Shared node address map_ in the Runtime section). + +**Invalidation responsibility.** +Staleness invalidation is the _router's_ responsibility, not the runtime's. +The router calls `invalidateRoute(route)` when it observes staleness indicators (e.g. `NOT_LEADER_OR_FOLLOWER`, `NOT_COORDINATOR`) in responses. +No background METADATA refresh is fired — the stale error is returned to the client unchanged, and the client sends its own METADATA request (standard Kafka behaviour), which repopulates the cache via the side-effect path. + +The alternative — having the runtime scan responses for staleness errors automatically — was rejected because it would require the runtime to deserialise an open-ended and growing set of response types. +Different error codes have different semantics (`NOT_LEADER_OR_FOLLOWER` vs `NOT_COORDINATOR` vs `FENCED_LEADER_EPOCH`). +Routers that don't use the topology cache shouldn't pay the deserialization cost. + +**Supporting types:** +* `PartitionInfo(VirtualNode leader, List replicas, List isr)` — full partition topology for follower-fetch / AZ-aware routing. +* `BrokerInfo(String host, int port, @Nullable String rack)` — broker metadata including rack assignment. + ### Configuration @@ -454,7 +541,7 @@ The runtime dispatches incoming requests in one of two modes: #### Nested router dispatch -When a router calls `sendRequestToNode()` on a route whose target is another router, the runtime dispatches to the nested router rather than forwarding directly to a backend. +When a router calls `sendRequest()` on a route whose target is another router, the runtime dispatches to the nested router rather than forwarding directly to a backend. The dispatch works as follows: 1. The runtime creates (or retrieves from a per-connection cache) a `Router` instance for the nested router. @@ -484,13 +571,44 @@ A router may send multiple requests (fan-out) and compose their responses before Different routes may respond at different times. The runtime uses a _response sequencer_ to ensure that responses are delivered to the client in the same order as the original client requests, regardless of the order in which fan-out responses arrive. +#### Topology cache population + +When a `TopologyService` has been created for a router level (via `RouterFactoryContext.topologyService()`), the runtime populates it from METADATA responses. +In `RoutingDecisionHandler.write()`, the processing order for each backend response is: + +1. `MetadataAddressCacher` extracts broker addresses (before node ID translation, using the route's `NodeIdMapping` to store addresses keyed by the outermost virtual ID). +2. `NodeIdResponseTranslator` translates all node IDs in-place from target-cluster IDs to virtual IDs. +3. `TopologyCache.updateFromMetadata()` updates the cache with translated (virtual) node IDs — partition leaders, replicas, ISR, broker info, and topicId→name mappings (if the cache exists for this router level). +4. The router's `CompletionStage` future is completed. + +This ordering guarantees that the topology cache reflects the response by the time the router's callback fires. + +#### Shared node address map + +Broker address resolution (`sharedNodeAddresses`) is shared per router level, backed by `ConcurrentHashMap` stored in `RouterChainFactory`. +This is necessary because the `TopologyCache` is shared: a leader virtual node ID cached from connection A's METADATA must be resolvable to a broker address on connection B. +If addresses were per-connection, connection B would find the leader in the topology cache (skipping METADATA), but then fail to resolve the leader's address — "Upstream address not yet known." + +The shared address map is populated from two sources: +* `RouterContextImpl.registerBootstrapAddresses()` — at handler installation time, adds bootstrap addresses for cluster-targeting routes. +* `MetadataAddressCacher.cacheIfMetadata()` — from METADATA responses, adds broker addresses learned from the broker list. + +#### TopologyService request sending + +The `TopologyServiceImpl` is shared per router level (created in `RouterChainFactory`). +Methods that may send requests (`ensureLeadersCached`, `discoverCoordinator`, `topicNames`) use a `RequestSender` functional interface, bound per-connection via `TopologyServiceImpl.bindRequestSender()`. +The binding happens in `RoutingDecisionHandler.dispatchDynamically()` before each `Router.onRequest()` call — the sender lambda captures the per-request `RouterContextImpl`. + +The `RequestSender` field is volatile. +This is safe because all calls to `TopologyService` methods happen on the event loop thread during `onRequest` processing, and `bindRequestSender` is called on the same thread before `onRequest`. + #### Node ID mapping implementation -As described in _Virtual node IDs_ above, the runtime translates between real target-cluster node IDs and virtual node IDs. +As described in _VirtualNode_ above, the runtime translates between real target-cluster node IDs and virtual node IDs. This is implemented by a `NodeIdMapping` abstraction (internal to the runtime) with two operations: * `toVirtual(route, targetNodeId)` — used when rewriting responses received from a target cluster (the runtime rewrites broker node IDs in `METADATA`, `PRODUCE`, `FIND_COORDINATOR`, and `DESCRIBE_CLUSTER` responses before they reach the router). -* `fromVirtual(virtualNodeId)` — used when the router calls `sendRequestToNode(virtualNodeId, ...)`, returning both the route and the target-cluster node ID. +* `fromVirtual(virtualNodeId)` — used internally by the runtime when the router calls `sendRequest(VirtualNode, ...)`. The runtime unwraps the `VirtualNode` to its encoded integer, calls `fromVirtual()`, and gets both the route and the target-cluster node ID. Routers never call `fromVirtual()` directly. The mapping must satisfy: for any route `r` and target node ID `t`, if `v = toVirtual(r, t)` then `fromVirtual(v)` returns `(r, t)`. The mapping must be able to recover the route from the virtual node ID alone. @@ -550,7 +668,7 @@ The runtime should respond with an appropriate Kafka error code, triggering the A shared mapping would require strong consistency across proxy instances to ensure they agree on the virtual node ID assignments. -Note: The current implementation uses a dedicated mapping and does not support shared mappings. Supporting shared mappings would require changes to the `NodeIdMapping` interface to allow `fromVirtual()` to recover the route. +Note: The current implementation uses a dedicated mapping and does not support shared mappings. The `VirtualNode` abstraction is designed to enable shared mappings in the future: a shared-mapping `VirtualNode` implementation could carry proxy instance identity, shard assignments, or other routing metadata internally, without changing the `sendRequest()` API that routers use. **Role-based mapping.** Virtual nodes correspond to per-route functional roles (e.g. "partition leader", "group coordinator"). @@ -570,8 +688,8 @@ Changing the mapping strategy (e.g. from dedicated to shared) is expected to be The Kafka protocol scopes `API_VERSIONS` to a single broker connection — a client sends `API_VERSIONS` each time it connects to a new broker. This means the proxy must forward `API_VERSIONS` to the specific broker the client believes it is talking to, not to an arbitrary broker. -When the client connects to a broker-specific endpoint, `virtualNodeId()` returns that broker's virtual node ID, allowing the router to forward `API_VERSIONS` to the correct broker via `sendRequestToNode()`. -When the client connects to a bootstrap address, `virtualNodeId()` returns empty, and the router should use `anyNodeId(route)` to send `API_VERSIONS` to an arbitrary broker on the route. +When the client connects to a broker-specific endpoint, `virtualNode()` returns that broker's `VirtualNode`, allowing the router to forward `API_VERSIONS` to the correct broker via `sendRequest()`. +When the client connects to a bootstrap address, `virtualNode()` returns empty, and the router should use `anyNode(route)` to send `API_VERSIONS` to an arbitrary broker on the route. With a shared mapping, the runtime would need to fan out `API_VERSIONS` to all brokers behind that virtual node and return the intersection of their version ranges. @@ -633,8 +751,8 @@ This cardinality is comparable to the existing per-filter metrics and should not ## Affected/not affected projects -* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResponse`, `CloseOrTerminalStage`, `TerminalStage`, `CloseStage`. -* `kroxylicious-runtime` — routing engine, configuration model (`RouterDefinition`, `RouteDefinition`, `TargetClusterDefinition`), graph validation, node ID mapping, response sequencing, metrics. +* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResponse`, `CloseOrTerminalStage`, `TerminalStage`, `CloseStage`, `VirtualNode`, `TopologyService`, `PartitionInfo`, `BrokerInfo`. +* `kroxylicious-runtime` — routing engine, configuration model (`RouterDefinition`, `RouteDefinition`, `TargetClusterDefinition`), graph validation, node ID mapping (`NodeIdMapping`, `BijectiveNodeIdMapping`, `IdentityNodeIdMapping`, `VirtualNodeImpl`), topology caching (`TopologyCache`, `TopologyServiceImpl`), response sequencing, metrics. * `kroxylicious-bom` — version management for any new modules. * Not affected: existing filters, KMS, authoriser API. The Kubernetes operator will need a separate update to support router configuration in CRDs. @@ -669,13 +787,19 @@ Having a single `clusterDefinitions` list as the authoritative catalogue of upst This section summarises the key design choices made in this proposal, for ease of reference. -* **Virtual node IDs** provide a uniform addressing scheme that insulates router authors from the node ID collision problem inherent in multi-cluster topologies. The runtime owns the mapping; routers work exclusively with virtual IDs and never need to translate. -* **Separate `virtualNodeId()` and `anyNodeId(route)` methods** distinguish between broker-specific connections (where the client connected to a specific broker endpoint) and bootstrap connections (where the client connected to a generic bootstrap address). This allows routers to correctly handle `API_VERSIONS` requests by forwarding them to the specific broker the client connected to, which is important during rolling upgrades where different brokers may run different Kafka versions. -* **Route derived from virtual node ID** — `sendRequestToNode()` takes only a virtual node ID, and the runtime derives the route via `NodeIdMapping.fromVirtual()`. This works for dedicated (one-to-one) mappings where each virtual node belongs to exactly one route. Shared (many-to-one) mappings would require the mapping to maintain additional state to recover the route, or be restricted such that each virtual node still maps to exactly one route. +* **`VirtualNode` as opaque type** provides a uniform addressing scheme that insulates router authors from the node ID collision problem inherent in multi-cluster topologies, and enables alternative networking models (proxy-as-broker) without changing the Router API. The runtime owns the mapping; routers work exclusively with `VirtualNode` instances and never need to know the underlying encoding. Routers never perform arithmetic on node IDs — they use them as opaque handles for map keys and `sendRequest()` arguments. +* **`nodeForId(int)` as wire-protocol bridge** converts integer node IDs from Kafka protocol response bodies into `VirtualNode` instances. This is retained permanently (not transitional) because routers will always need to interpret integers from protocol messages — for METADATA response merging, FIND_COORDINATOR responses, and future protocol extensions. Removing it would mean every such use case needs a dedicated method on `RouterContext` or `TopologyService`, which does not scale. +* **Separate `virtualNode()` and `anyNode(route)` methods** distinguish between broker-specific connections (where the client connected to a specific broker endpoint) and bootstrap connections (where the client connected to a generic bootstrap address). This allows routers to correctly handle `API_VERSIONS` requests by forwarding them to the specific broker the client connected to, which is important during rolling upgrades where different brokers may run different Kafka versions. +* **Route derived from `VirtualNode`** — `sendRequest()` takes only a `VirtualNode`, and the runtime derives the route by unwrapping to the internal integer and calling `NodeIdMapping.fromVirtual()`. This works for dedicated (one-to-one) mappings where each virtual node belongs to exactly one route. Shared (many-to-one) mappings encode the routing metadata within the `VirtualNode` implementation, without changing the `sendRequest()` API. * **Per-connection `Router` instances** allow per-connection state (caches, sessions) without synchronisation. Shared state lives in the `RouterFactory`'s initialisation data, which must be thread-safe. * **Builder pattern for response construction** follows the same fluent interface pattern as `FilterResult`, where outcomes are constructed via `RouterContext` builder methods rather than by directly instantiating result types. This allows the API to evolve (e.g. adding new builder stages) without breaking existing implementations. -* **All requests addressed by virtual node ID.** Routers must explicitly obtain a virtual node ID (via `virtualNodeId()`, `anyNodeId(route)`, or from a previous response) before sending a request. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. -* **`topicName(Uuid)` for synchronous topic ID resolution** allows routers to resolve topic IDs to names without relying on wire-object enrichment. This is necessary for Kafka APIs (such as `SHARE_FETCH`) that have only a topic ID field. The runtime guarantees the cache is warm before `onRequest()` is called. +* **All requests addressed by `VirtualNode`.** Routers must explicitly obtain a `VirtualNode` (via `virtualNode()`, `anyNode(route)`, `nodeForId(int)`, or from `TopologyService`) before sending a request. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. +* **`TopologyService` is opt-in** via `RouterFactoryContext.topologyService()`. The runtime creates the underlying cache lazily on first call. Simple routers (subject-based, client-ID-based) that never call it pay no cost — no cache, no topology-related processing. This avoids a one-size-fits-all design where all routers pay for topology caching. +* **Side-effect cache population** — the topology cache is populated from METADATA responses in `RoutingDecisionHandler.write()`, after node ID translation, before the router's `CompletionStage` completes. This is simpler and more automatic than giving `TopologyService` full request-sending capability for all cache operations. +* **Coarse invalidation** via `invalidateRoute(route)` rather than fine-grained `invalidateLeader()`, `invalidateCoordinator()`, `invalidateNode()`. A single method clears partition info, coordinators, and broker info for a route (but not topic ID→name mappings, which are stable within a cluster). Over-invalidation is acceptable because the cache is repopulated cheaply from the client's next METADATA request. This avoids an ever-growing set of invalidation methods as more cached entity types are added. +* **Router-driven invalidation** rather than runtime-driven. The runtime would need to deserialise an open-ended and growing set of response types to scan for staleness error codes. Different error codes have different semantics. Routers that don't use the topology cache shouldn't pay the deserialisation cost. +* **Shared node address map** — broker address resolution is per-router-level (`ConcurrentHashMap`), not per-connection. Required because the shared topology cache means a leader virtual node ID cached from one connection's METADATA must be resolvable to a broker address on another connection. +* **`topicName(Uuid)` for synchronous topic ID resolution** allows routers to resolve topic IDs to names without relying on wire-object enrichment. This is necessary for Kafka APIs (such as `SHARE_FETCH`) that have only a topic ID field. The runtime guarantees the cache is warm before `onRequest()` is called. `TopologyService` also offers `topicNames(Set)` as an async batched alternative. * **`routeNames()` in `RouterFactoryContext`** allows router factories to validate at initialization time that route names referenced in their plugin configuration actually exist, providing early feedback on configuration errors. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. From 5bf815090eadb5a0e1d4f70e4e90929978ed9aaf Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 15 Jun 2026 21:35:52 +0000 Subject: [PATCH 19/30] docs: update coordinator caching to side-effect model MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Coordinators are now cached via the same side-effect path as METADATA — PendingResponse carries request-side context (keyType, key) that the runtime uses alongside the FIND_COORDINATOR response to populate the TopologyCache. No explicit putCoordinator call needed from the router. Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 8f546ee0..b364ac7b 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -374,9 +374,9 @@ When any request sent via `RouterContext.sendRequest()` produces a METADATA resp This means: after a METADATA request's future completes, the cache is guaranteed to reflect that response. Routers that need to warm the cache for uncached topics send METADATA requests themselves (or via `ensureLeadersCached()`), and the cache is populated as a side effect of the response flow. -Coordinator caching works differently: the FIND_COORDINATOR response lacks `keyType`, and pre-v4 responses lack the `key` field entirely. -So coordinators cannot be cached from the response alone. -The `TopologyServiceImpl.discoverCoordinator()` method caches them explicitly using request-side context (the key type and key from the original request). +Coordinator caching uses the same side-effect model, with one twist: the FIND_COORDINATOR response lacks `keyType`, and pre-v4 responses lack the `key` field entirely. +To solve this, `PendingResponse` carries a `CoordinatorRequestContext(keyType, key)` extracted from the FIND_COORDINATOR request. +When the response arrives in `RoutingDecisionHandler.write()`, the runtime uses this request-side context alongside the response to populate the cache — `TopologyCache.updateFromFindCoordinator()` matches coordinator keys from the request with node IDs from the response. Coordinators are keyed by `(route, keyType, key)` — not by route alone, fixing a limitation where different consumer groups on the same route could collide. Only METADATA responses populate partition and broker data. @@ -578,7 +578,9 @@ In `RoutingDecisionHandler.write()`, the processing order for each backend respo 1. `MetadataAddressCacher` extracts broker addresses (before node ID translation, using the route's `NodeIdMapping` to store addresses keyed by the outermost virtual ID). 2. `NodeIdResponseTranslator` translates all node IDs in-place from target-cluster IDs to virtual IDs. -3. `TopologyCache.updateFromMetadata()` updates the cache with translated (virtual) node IDs — partition leaders, replicas, ISR, broker info, and topicId→name mappings (if the cache exists for this router level). +3. Topology cache update (if the cache exists for this router level): + - For METADATA responses: `TopologyCache.updateFromMetadata()` updates partition leaders, replicas, ISR, broker info, and topicId→name mappings. + - For FIND_COORDINATOR responses: `TopologyCache.updateFromFindCoordinator()` uses the `CoordinatorRequestContext` (keyType, key) carried by `PendingResponse` from the original request. 4. The router's `CompletionStage` future is completed. This ordering guarantees that the topology cache reflects the response by the time the router's callback fires. From 3394c3c11734209240a119812ba5e33568f9cf7e Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 15 Jun 2026 22:13:29 +0000 Subject: [PATCH 20/30] docs: replace sync cache queries with discovery result types TopologyService now has three symmetric discovery methods: - leaders() -> CompletionStage - coordinators() -> CompletionStage - topicNames() -> CompletionStage> Removes leaderOf(), coordinatorOf(), ensureLeadersCached(), and discoverCoordinator(). Documents the rationale: sync cache queries are unsafe because the shared cache can be invalidated between discovery and query. Self-contained snapshots eliminate this race. Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 52 ++++++++++++++++++++++++------------ 1 file changed, 35 insertions(+), 17 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index b364ac7b..dde7b7b0 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -337,37 +337,52 @@ Routers that never call `topologyService()` pay no cost — no cache is created, ```java interface TopologyService { + // Discovery (async, may send requests, return self-contained results) + CompletionStage leaders(Map> topicsByRoute); + CompletionStage coordinators(String route, byte keyType, Set keys); CompletionStage> topicNames(Set topicIds); - Optional leaderOf(String topicName, int partitionIndex); - CompletionStage ensureLeadersCached(Map> topicsByRoute); - - Optional coordinatorOf(String route, byte keyType, String key); - CompletionStage discoverCoordinator(String route, byte keyType, String key); - - Optional partitionInfoFor(String topicName, int partitionIndex); + // Supplementary lookups (sync, best-effort from cache) + Optional partitionInfo(String topicName, int partitionIndex); Optional brokerInfo(VirtualNode node); + // Invalidation void invalidateRoute(String route); } + +interface PartitionLeaders { + Optional leaderOf(String topicName, int partitionIndex); +} + +interface Coordinators { + Optional coordinatorFor(String key); +} ``` The service provides three categories of methods: -**Synchronous reads** — `leaderOf()`, `coordinatorOf()`, `partitionInfoFor()`, `brokerInfo()` — query the cache and return immediately. -If the data is not cached, they return empty. -Callers should warm the cache first (via `ensureLeadersCached()`, `discoverCoordinator()`, or by sending METADATA directly). - -**Asynchronous operations** — `ensureLeadersCached()`, `discoverCoordinator()`, `topicNames()` — may send requests internally to warm the cache. -`ensureLeadersCached()` batches all uncached topics into one METADATA request per route. -`discoverCoordinator()` sends METADATA (if needed) then FIND_COORDINATOR. -`topicNames()` resolves topic IDs to names, batching cache misses into a single METADATA request. +**Discovery** — `leaders()`, `coordinators()`, `topicNames()` — are async and return self-contained result objects. +They may send requests internally (METADATA for leaders and topic names, FIND_COORDINATOR for coordinators). +The results are valid immediately upon completion and do not require further cache queries. +`leaders()` batches all uncached topics into one METADATA request per route and returns a `PartitionLeaders` snapshot. +`coordinators()` sends METADATA (if needed) then FIND_COORDINATOR and returns a `Coordinators` snapshot; it supports batched lookup matching the FIND_COORDINATOR v4+ protocol. +`topicNames()` resolves topic IDs to names, batching cache misses. These methods use a `RequestSender` bound per-connection by the runtime (see _Runtime_ below). +**Supplementary lookups** — `partitionInfo()`, `brokerInfo()` — query the cache synchronously and return immediately. +These are for use cases like follower-fetch / AZ-aware routing where the router needs replica or rack information beyond what `PartitionLeaders` provides. +If they return empty, the router can fall back to the leader from `PartitionLeaders`. + **Invalidation** — `invalidateRoute(route)` performs coarse invalidation: clears all partition info (leaders, replicas, ISR), coordinators, and broker info for a route. Topic ID→name mappings are _not_ cleared (they are stable within a cluster — a topic ID always maps to the same name). Over-invalidation is acceptable because the cache is repopulated cheaply from the client's next METADATA request. +**Why discovery methods return result objects, not void.** +An earlier design had `ensureLeadersCached()` returning `CompletionStage` (warming the cache as a side effect) and separate synchronous `leaderOf()`/`coordinatorOf()` methods for querying the cache. +This was rejected because the synchronous methods are unsafe: the cache is shared across connections and can be invalidated at any time by another connection calling `invalidateRoute()`. +The only moment a leader is guaranteed to be in the cache is immediately after the discovery future completes — and even then, another thread could invalidate between completion and the synchronous call. +Returning a self-contained `PartitionLeaders` snapshot eliminates this race: the router uses the snapshot directly, not the cache. + **Cache population model.** The cache is populated as a _side effect_ of METADATA responses flowing through the routing pipeline. When any request sent via `RouterContext.sendRequest()` produces a METADATA response, the runtime intercepts the response in `RoutingDecisionHandler.write()` (after node ID translation) and updates the topology cache _before_ completing the router's `CompletionStage`. @@ -397,6 +412,8 @@ Different error codes have different semantics (`NOT_LEADER_OR_FOLLOWER` vs `NOT Routers that don't use the topology cache shouldn't pay the deserialization cost. **Supporting types:** +* `PartitionLeaders` — snapshot of partition leader assignments, returned by `leaders()`. Has `leaderOf(topicName, partitionIndex)`. +* `Coordinators` — snapshot of coordinator assignments, returned by `coordinators()`. Has `coordinatorFor(key)`. The key type is implicit (specified in the `coordinators()` call). * `PartitionInfo(VirtualNode leader, List replicas, List isr)` — full partition topology for follower-fetch / AZ-aware routing. * `BrokerInfo(String host, int port, @Nullable String rack)` — broker metadata including rack assignment. @@ -753,7 +770,7 @@ This cardinality is comparable to the existing per-filter metrics and should not ## Affected/not affected projects -* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResponse`, `CloseOrTerminalStage`, `TerminalStage`, `CloseStage`, `VirtualNode`, `TopologyService`, `PartitionInfo`, `BrokerInfo`. +* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResponse`, `CloseOrTerminalStage`, `TerminalStage`, `CloseStage`, `VirtualNode`, `TopologyService`, `PartitionLeaders`, `Coordinators`, `PartitionInfo`, `BrokerInfo`. * `kroxylicious-runtime` — routing engine, configuration model (`RouterDefinition`, `RouteDefinition`, `TargetClusterDefinition`), graph validation, node ID mapping (`NodeIdMapping`, `BijectiveNodeIdMapping`, `IdentityNodeIdMapping`, `VirtualNodeImpl`), topology caching (`TopologyCache`, `TopologyServiceImpl`), response sequencing, metrics. * `kroxylicious-bom` — version management for any new modules. * Not affected: existing filters, KMS, authoriser API. The Kubernetes operator will need a separate update to support router configuration in CRDs. @@ -797,7 +814,8 @@ This section summarises the key design choices made in this proposal, for ease o * **Builder pattern for response construction** follows the same fluent interface pattern as `FilterResult`, where outcomes are constructed via `RouterContext` builder methods rather than by directly instantiating result types. This allows the API to evolve (e.g. adding new builder stages) without breaking existing implementations. * **All requests addressed by `VirtualNode`.** Routers must explicitly obtain a `VirtualNode` (via `virtualNode()`, `anyNode(route)`, `nodeForId(int)`, or from `TopologyService`) before sending a request. This forces router authors to think about broker targeting, avoiding a class of bugs where a broker-specific API is mistakenly sent to an arbitrary broker. * **`TopologyService` is opt-in** via `RouterFactoryContext.topologyService()`. The runtime creates the underlying cache lazily on first call. Simple routers (subject-based, client-ID-based) that never call it pay no cost — no cache, no topology-related processing. This avoids a one-size-fits-all design where all routers pay for topology caching. -* **Side-effect cache population** — the topology cache is populated from METADATA responses in `RoutingDecisionHandler.write()`, after node ID translation, before the router's `CompletionStage` completes. This is simpler and more automatic than giving `TopologyService` full request-sending capability for all cache operations. +* **Side-effect cache population** — the topology cache is populated from METADATA and FIND_COORDINATOR responses in `RoutingDecisionHandler.write()`, after node ID translation, before the router's `CompletionStage` completes. Discovery methods (`leaders()`, `coordinators()`) return self-contained snapshot objects (`PartitionLeaders`, `Coordinators`) that read from the now-warm cache. The snapshots are safe to use across async callback boundaries. +* **Discovery methods return snapshots, not void** — an earlier design had `ensureLeadersCached()` returning `CompletionStage` with separate synchronous `leaderOf()`/`coordinatorOf()` methods. This was rejected because the synchronous methods are unsafe: the shared cache can be invalidated between the discovery future completing and the synchronous query. Returning a self-contained snapshot eliminates this race. It also makes the three discovery methods symmetric: `leaders()`, `coordinators()`, `topicNames()` all return self-contained results. * **Coarse invalidation** via `invalidateRoute(route)` rather than fine-grained `invalidateLeader()`, `invalidateCoordinator()`, `invalidateNode()`. A single method clears partition info, coordinators, and broker info for a route (but not topic ID→name mappings, which are stable within a cluster). Over-invalidation is acceptable because the cache is repopulated cheaply from the client's next METADATA request. This avoids an ever-growing set of invalidation methods as more cached entity types are added. * **Router-driven invalidation** rather than runtime-driven. The runtime would need to deserialise an open-ended and growing set of response types to scan for staleness error codes. Different error codes have different semantics. Routers that don't use the topology cache shouldn't pay the deserialisation cost. * **Shared node address map** — broker address resolution is per-router-level (`ConcurrentHashMap`), not per-connection. Required because the shared topology cache means a leader virtual node ID cached from one connection's METADATA must be resolvable to a broker address on another connection. From 3599608e7d343f27b6fda4f6708e1a302f0b4eaf Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Mon, 15 Jun 2026 22:57:05 +0000 Subject: [PATCH 21/30] docs: remove topicName(Uuid) from RouterContext, use TopologyService Topic ID resolution moves from the synchronous RouterContext.topicName() to the async TopologyService.topicNames(Set), consistent with leaders() and coordinators(). Documents the rationale: the synchronous method silently swallowed resolution errors and was inconsistent with the async discovery pattern. Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index dde7b7b0..7d60b282 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -202,8 +202,6 @@ interface RouterContext { Subject authenticatedSubject(); - String topicName(Uuid topicId); - CloseOrTerminalStage respondWith(ApiMessage body); CloseOrTerminalStage respondWith(ResponseHeaderData header, ApiMessage body); @@ -259,12 +257,6 @@ If no authentication has occurred, the subject will be anonymous. Correct placement of SASL plugins in the topology is the operator's responsibility; a future proposal may add runtime validation of SASL plugin placement. -**`topicName(topicId)`** resolves a topic ID to its topic name synchronously. -The runtime guarantees that all topic IDs present in the current request have been resolved before `Router.onRequest()` is called, so this method returns immediately from a per-connection cache. -The cache is populated by an internal filter that sends `METADATA` requests on cache miss. -This is necessary for Kafka APIs (such as `SHARE_FETCH`) that have only a topic ID field with no topic name property. -Returns `null` if the topic ID could not be resolved (e.g. the topic was deleted). - **Response builder methods** — `respondWith()`, `respondWithError()`, and `respondWithoutReply()` — follow a fluent builder pattern consistent with the `Filter` API. Rather than constructing `RouterResult` subtypes directly, routers use the builder to construct responses: * `context.respondWith(body).build()` — delivers a response to the client @@ -819,7 +811,7 @@ This section summarises the key design choices made in this proposal, for ease o * **Coarse invalidation** via `invalidateRoute(route)` rather than fine-grained `invalidateLeader()`, `invalidateCoordinator()`, `invalidateNode()`. A single method clears partition info, coordinators, and broker info for a route (but not topic ID→name mappings, which are stable within a cluster). Over-invalidation is acceptable because the cache is repopulated cheaply from the client's next METADATA request. This avoids an ever-growing set of invalidation methods as more cached entity types are added. * **Router-driven invalidation** rather than runtime-driven. The runtime would need to deserialise an open-ended and growing set of response types to scan for staleness error codes. Different error codes have different semantics. Routers that don't use the topology cache shouldn't pay the deserialisation cost. * **Shared node address map** — broker address resolution is per-router-level (`ConcurrentHashMap`), not per-connection. Required because the shared topology cache means a leader virtual node ID cached from one connection's METADATA must be resolvable to a broker address on another connection. -* **`topicName(Uuid)` for synchronous topic ID resolution** allows routers to resolve topic IDs to names without relying on wire-object enrichment. This is necessary for Kafka APIs (such as `SHARE_FETCH`) that have only a topic ID field. The runtime guarantees the cache is warm before `onRequest()` is called. `TopologyService` also offers `topicNames(Set)` as an async batched alternative. +* **Topic ID resolution via `TopologyService.topicNames(Set)`** is async and batched, consistent with `leaders()` and `coordinators()`. An earlier design had a synchronous `topicName(Uuid)` on `RouterContext`, preloaded by an internal filter before `onRequest()`. This was removed because it silently swallowed resolution errors (returning null for both "not yet resolved" and "topic deleted") and was inconsistent with the async discovery pattern. Routers that need topic names collect IDs from the request, call `topicNames()`, and pass the resolved map to decomposers. * **`routeNames()` in `RouterFactoryContext`** allows router factories to validate at initialization time that route names referenced in their plugin configuration actually exist, providing early feedback on configuration errors. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. From 6c8fd4def42547ab4f3f75899fe22ac92335a803 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Tue, 16 Jun 2026 01:15:51 +0000 Subject: [PATCH 22/30] docs: add allowSharedClusterTargets() and disjoint cluster validation Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 7d60b282..1ba3dec1 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -180,6 +180,7 @@ Because `Router` instances may be created on different event loop threads, the i * `routeNames()` — the set of route names declared in the router definition. This allows router factories to validate that route names referenced in their plugin configuration actually exist. * `pluginInstance()` / `pluginImplementationNames()` — access to the plugin registry. * `topologyService()` — returns a `TopologyService` for opt-in topology caching (see _TopologyService_ below). The runtime creates the underlying cache lazily on first call. Routers that never call this method pay no cost. The returned service should be stored in the factory's initialisation data so it survives connection reconnects. +* `allowSharedClusterTargets()` — declares that this router supports multiple routes resolving to the same cluster. By default the runtime rejects such configurations at startup because most routers assume each route is a distinct destination. Routers that handle this (e.g. aliasing the same cluster under different prefixes for migration) call this during `initialize()` to suppress the check. The check is transitive: if route A targets cluster X directly and route B targets a nested router that eventually reaches cluster X, the outer router's routes are considered overlapping. #### `RouterContext` @@ -762,7 +763,7 @@ This cardinality is comparable to the existing per-filter metrics and should not ## Affected/not affected projects -* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResponse`, `CloseOrTerminalStage`, `TerminalStage`, `CloseStage`, `VirtualNode`, `TopologyService`, `PartitionLeaders`, `Coordinators`, `PartitionInfo`, `BrokerInfo`. +* `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResponse`, `CloseOrTerminalStage`, `TerminalStage`, `CloseStage`; new `io.kroxylicious.proxy.topology` package containing `VirtualNode`, `TopologyService`, `PartitionLeaders`, `Coordinators`, `PartitionInfo`, `BrokerInfo`. * `kroxylicious-runtime` — routing engine, configuration model (`RouterDefinition`, `RouteDefinition`, `TargetClusterDefinition`), graph validation, node ID mapping (`NodeIdMapping`, `BijectiveNodeIdMapping`, `IdentityNodeIdMapping`, `VirtualNodeImpl`), topology caching (`TopologyCache`, `TopologyServiceImpl`), response sequencing, metrics. * `kroxylicious-bom` — version management for any new modules. * Not affected: existing filters, KMS, authoriser API. The Kubernetes operator will need a separate update to support router configuration in CRDs. @@ -812,6 +813,7 @@ This section summarises the key design choices made in this proposal, for ease o * **Router-driven invalidation** rather than runtime-driven. The runtime would need to deserialise an open-ended and growing set of response types to scan for staleness error codes. Different error codes have different semantics. Routers that don't use the topology cache shouldn't pay the deserialisation cost. * **Shared node address map** — broker address resolution is per-router-level (`ConcurrentHashMap`), not per-connection. Required because the shared topology cache means a leader virtual node ID cached from one connection's METADATA must be resolvable to a broker address on another connection. * **Topic ID resolution via `TopologyService.topicNames(Set)`** is async and batched, consistent with `leaders()` and `coordinators()`. An earlier design had a synchronous `topicName(Uuid)` on `RouterContext`, preloaded by an internal filter before `onRequest()`. This was removed because it silently swallowed resolution errors (returning null for both "not yet resolved" and "topic deleted") and was inconsistent with the async discovery pattern. Routers that need topic names collect IDs from the request, call `topicNames()`, and pass the resolved map to decomposers. +* **Routes must target distinct clusters by default.** The runtime rejects configurations where two routes in the same router resolve (directly or transitively) to the same cluster, because most routers assume each route is a distinct destination. Router factories that handle shared cluster targets (e.g. aliasing the same cluster under different prefixes) call `allowSharedClusterTargets()` during `initialize()` to suppress the check. The check runs in `RouterChainFactory` after factory initialization, not in `RouterGraphValidator`, because it depends on the factory's runtime declaration. * **`routeNames()` in `RouterFactoryContext`** allows router factories to validate at initialization time that route names referenced in their plugin configuration actually exist, providing early feedback on configuration errors. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. From 77fd121e90029883464b719e9b8cd4274a40e4d2 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Tue, 16 Jun 2026 21:18:54 +0000 Subject: [PATCH 23/30] docs: document TopologyCache authorization safety The shared topology cache is not scoped by authenticated subject. Document why this is safe (additive/union semantics, routing-only use, backend ACL enforcement) and clarify that router authors must not use the cache for authorization decisions. Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 1ba3dec1..bd1ffb27 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -395,6 +395,19 @@ The cache is shared per router level (not per connection), backed by `Concurrent All connections through the same router level share the same topology view. This is efficient (no per-connection duplication) and necessary for correctness: a leader discovered by connection A must be usable by connection B (see _Shared node address map_ in the Runtime section). +**Authorization and cache scope.** +The cache is not scoped by authenticated subject. +When Kafka brokers filter METADATA responses based on ACLs, different connections may receive different subsets of topics. +Because the cache is shared, it converges toward the _union_ of all connections' views: each METADATA response adds or updates entries for topics present in the response, but does not remove entries for topics absent from the response. + +This is safe because the cache is a _routing optimization_, not an access-control mechanism: +* The cache stores _where_ to send requests (partition leaders, coordinator nodes, broker addresses), not _whether_ a client is authorized. +* Backend brokers enforce ACLs on every request. A client that resolves a leader from the cache but lacks permission to produce or fetch will receive an authorization error from the broker, exactly as it would without the proxy. +* Cache misses trigger discovery requests on the current connection's `RequestSender`, which carries that connection's authentication context. A low-privilege connection's filtered METADATA response adds a subset of entries without corrupting entries populated by other connections. + +Router authors must not use the topology cache for authorization decisions. +Routers should use the cache only for routing (leader selection, coordinator discovery, broker address resolution) and rely on backend broker ACL enforcement for access control. + **Invalidation responsibility.** Staleness invalidation is the _router's_ responsibility, not the runtime's. The router calls `invalidateRoute(route)` when it observes staleness indicators (e.g. `NOT_LEADER_OR_FOLLOWER`, `NOT_COORDINATOR`) in responses. From 5a75a7b9928f31ddcede66614e2e98afae8941bf Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Tue, 16 Jun 2026 21:57:08 +0000 Subject: [PATCH 24/30] docs: annotate internal runtime classes in routing API proposal Mark references to internal runtime classes (VirtualNodeImpl, RoutingDecisionHandler, PendingResponse, CoordinatorRequestContext, TopologyCache, RouterContextImpl, TopologyServiceImpl, RouterChainFactory, MetadataAddressCacher, NodeIdResponseTranslator, RouterGraphValidator) so readers can distinguish them from public API types. Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index bd1ffb27..17396f53 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -96,7 +96,7 @@ The runtime transparently translates between `VirtualNode` and real node IDs at The integer-based port-per-broker networking model encodes both route and target broker into a single `int` via a bijective formula. An alternative networking model — where proxy instances act as brokers with their own identities and shard-based ownership — needs richer routing information that cannot be encoded in a single integer. `VirtualNode` hides this difference: the same router code works with either networking model. -The runtime implementation (`VirtualNodeImpl`) wraps the encoded integer for the current model; a future implementation could carry proxy instance identity, shard keys, or other routing metadata. +The runtime's internal class `VirtualNodeImpl` wraps the encoded integer for the current model; a future implementation could carry proxy instance identity, shard keys, or other routing metadata. **`nodeForId(int)` — the wire-protocol bridge.** Kafka protocol messages (METADATA responses, FIND_COORDINATOR responses, etc.) carry node IDs as integers. @@ -378,13 +378,13 @@ Returning a self-contained `PartitionLeaders` snapshot eliminates this race: the **Cache population model.** The cache is populated as a _side effect_ of METADATA responses flowing through the routing pipeline. -When any request sent via `RouterContext.sendRequest()` produces a METADATA response, the runtime intercepts the response in `RoutingDecisionHandler.write()` (after node ID translation) and updates the topology cache _before_ completing the router's `CompletionStage`. +When any request sent via `RouterContext.sendRequest()` produces a METADATA response, the runtime intercepts the response in `RoutingDecisionHandler.write()` (an internal class; after node ID translation) and updates the topology cache _before_ completing the router's `CompletionStage`. This means: after a METADATA request's future completes, the cache is guaranteed to reflect that response. Routers that need to warm the cache for uncached topics send METADATA requests themselves (or via `ensureLeadersCached()`), and the cache is populated as a side effect of the response flow. Coordinator caching uses the same side-effect model, with one twist: the FIND_COORDINATOR response lacks `keyType`, and pre-v4 responses lack the `key` field entirely. -To solve this, `PendingResponse` carries a `CoordinatorRequestContext(keyType, key)` extracted from the FIND_COORDINATOR request. -When the response arrives in `RoutingDecisionHandler.write()`, the runtime uses this request-side context alongside the response to populate the cache — `TopologyCache.updateFromFindCoordinator()` matches coordinator keys from the request with node IDs from the response. +To solve this, `PendingResponse` (an internal class) carries a `CoordinatorRequestContext(keyType, key)` (also internal) extracted from the FIND_COORDINATOR request. +When the response arrives in `RoutingDecisionHandler.write()`, the runtime uses this request-side context alongside the response to populate the cache — the internal `TopologyCache.updateFromFindCoordinator()` method matches coordinator keys from the request with node IDs from the response. Coordinators are keyed by `(route, keyType, key)` — not by route alone, fixing a limitation where different consumer groups on the same route could collide. Only METADATA responses populate partition and broker data. @@ -568,7 +568,7 @@ When a router calls `sendRequest()` on a route whose target is another router, t The dispatch works as follows: 1. The runtime creates (or retrieves from a per-connection cache) a `Router` instance for the nested router. -2. A new `RouterContextImpl` is constructed for the nested router, with its own `NodeIdMapping`, routes, and bootstrap virtual node IDs. +2. A new `RouterContextImpl` (the internal implementation of `RouterContext`) is constructed for the nested router, with its own `NodeIdMapping`, routes, and bootstrap virtual node IDs. The nested context shares the same Netty client channel, response sequencer, correlation ID allocator, and metrics as the outer context. Its forwarder callbacks wrap the outer forwarders to translate virtual node IDs from the nested space to the outermost space (see _Per-router scoping and nested dispatch_ above). 3. The runtime invokes `nestedRouter.onRequest()` with the nested context. @@ -599,8 +599,8 @@ The runtime uses a _response sequencer_ to ensure that responses are delivered t When a `TopologyService` has been created for a router level (via `RouterFactoryContext.topologyService()`), the runtime populates it from METADATA responses. In `RoutingDecisionHandler.write()`, the processing order for each backend response is: -1. `MetadataAddressCacher` extracts broker addresses (before node ID translation, using the route's `NodeIdMapping` to store addresses keyed by the outermost virtual ID). -2. `NodeIdResponseTranslator` translates all node IDs in-place from target-cluster IDs to virtual IDs. +1. `MetadataAddressCacher` (an internal class) extracts broker addresses (before node ID translation, using the route's `NodeIdMapping` to store addresses keyed by the outermost virtual ID). +2. `NodeIdResponseTranslator` (an internal class) translates all node IDs in-place from target-cluster IDs to virtual IDs. 3. Topology cache update (if the cache exists for this router level): - For METADATA responses: `TopologyCache.updateFromMetadata()` updates partition leaders, replicas, ISR, broker info, and topicId→name mappings. - For FIND_COORDINATOR responses: `TopologyCache.updateFromFindCoordinator()` uses the `CoordinatorRequestContext` (keyType, key) carried by `PendingResponse` from the original request. @@ -620,7 +620,7 @@ The shared address map is populated from two sources: #### TopologyService request sending -The `TopologyServiceImpl` is shared per router level (created in `RouterChainFactory`). +The internal `TopologyServiceImpl` is shared per router level (created in the internal `RouterChainFactory`). Methods that may send requests (`ensureLeadersCached`, `discoverCoordinator`, `topicNames`) use a `RequestSender` functional interface, bound per-connection via `TopologyServiceImpl.bindRequestSender()`. The binding happens in `RoutingDecisionHandler.dispatchDynamically()` before each `Router.onRequest()` call — the sender lambda captures the per-request `RouterContextImpl`. @@ -777,7 +777,7 @@ This cardinality is comparable to the existing per-filter metrics and should not ## Affected/not affected projects * `kroxylicious-api` — new `io.kroxylicious.proxy.router` package containing `Router`, `RouterFactory`, `RouterFactoryContext`, `RouterContext`, `RouterResponse`, `CloseOrTerminalStage`, `TerminalStage`, `CloseStage`; new `io.kroxylicious.proxy.topology` package containing `VirtualNode`, `TopologyService`, `PartitionLeaders`, `Coordinators`, `PartitionInfo`, `BrokerInfo`. -* `kroxylicious-runtime` — routing engine, configuration model (`RouterDefinition`, `RouteDefinition`, `TargetClusterDefinition`), graph validation, node ID mapping (`NodeIdMapping`, `BijectiveNodeIdMapping`, `IdentityNodeIdMapping`, `VirtualNodeImpl`), topology caching (`TopologyCache`, `TopologyServiceImpl`), response sequencing, metrics. +* `kroxylicious-runtime` — routing engine, configuration model (`RouterDefinition`, `RouteDefinition`, `TargetClusterDefinition`), graph validation, node ID mapping (internal classes: `NodeIdMapping`, `BijectiveNodeIdMapping`, `IdentityNodeIdMapping`, `VirtualNodeImpl`), topology caching (internal classes: `TopologyCache`, `TopologyServiceImpl`), response sequencing, metrics. * `kroxylicious-bom` — version management for any new modules. * Not affected: existing filters, KMS, authoriser API. The Kubernetes operator will need a separate update to support router configuration in CRDs. @@ -826,7 +826,7 @@ This section summarises the key design choices made in this proposal, for ease o * **Router-driven invalidation** rather than runtime-driven. The runtime would need to deserialise an open-ended and growing set of response types to scan for staleness error codes. Different error codes have different semantics. Routers that don't use the topology cache shouldn't pay the deserialisation cost. * **Shared node address map** — broker address resolution is per-router-level (`ConcurrentHashMap`), not per-connection. Required because the shared topology cache means a leader virtual node ID cached from one connection's METADATA must be resolvable to a broker address on another connection. * **Topic ID resolution via `TopologyService.topicNames(Set)`** is async and batched, consistent with `leaders()` and `coordinators()`. An earlier design had a synchronous `topicName(Uuid)` on `RouterContext`, preloaded by an internal filter before `onRequest()`. This was removed because it silently swallowed resolution errors (returning null for both "not yet resolved" and "topic deleted") and was inconsistent with the async discovery pattern. Routers that need topic names collect IDs from the request, call `topicNames()`, and pass the resolved map to decomposers. -* **Routes must target distinct clusters by default.** The runtime rejects configurations where two routes in the same router resolve (directly or transitively) to the same cluster, because most routers assume each route is a distinct destination. Router factories that handle shared cluster targets (e.g. aliasing the same cluster under different prefixes) call `allowSharedClusterTargets()` during `initialize()` to suppress the check. The check runs in `RouterChainFactory` after factory initialization, not in `RouterGraphValidator`, because it depends on the factory's runtime declaration. +* **Routes must target distinct clusters by default.** The runtime rejects configurations where two routes in the same router resolve (directly or transitively) to the same cluster, because most routers assume each route is a distinct destination. Router factories that handle shared cluster targets (e.g. aliasing the same cluster under different prefixes) call `allowSharedClusterTargets()` during `initialize()` to suppress the check. The check runs in the internal `RouterChainFactory` after factory initialization, not in the internal `RouterGraphValidator`, because it depends on the factory's runtime declaration. * **`routeNames()` in `RouterFactoryContext`** allows router factories to validate at initialization time that route names referenced in their plugin configuration actually exist, providing early feedback on configuration errors. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. From 38a028909e507423260dd0141f71e5b14d6c852e Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 17 Jun 2026 10:00:05 +1200 Subject: [PATCH 25/30] RouterFactory scope Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 1 + 1 file changed, 1 insertion(+) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 17396f53..0fc033cd 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -154,6 +154,7 @@ Implementations should release any per-connection resources (session caches, etc #### `RouterFactory` `RouterFactory` manages the lifecycle of `Router` instances, analogous to `FilterFactory` for filters. +Like `FilterFactory`, each instance is scoped to a single virtual cluster. ```java interface RouterFactory { From 32036a7082216fad1f551f8fce4ac573459cff5b Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 17 Jun 2026 11:35:26 +1200 Subject: [PATCH 26/30] Keith's suggestions Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 0fc033cd..3ec5c1ad 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -256,7 +256,7 @@ For shared (many-to-one) mappings — where a single virtual node may serve brok `sessionId()` returns a string that uniquely identifies the connection with the Kafka client. It will have the same value for all invocations of `onRequest()` which happen for that client connection, both for the same router over time, and different routers in a topology. `authenticatedSubject()` returns the client's `Subject` established by mTLS or SASL processing on the VC filter chain (e.g. a SASL termination filter). If no authentication has occurred, the subject will be anonymous. -Correct placement of SASL plugins in the topology is the operator's responsibility; +Correct placement of SASL plugins in the topology is the administrator's responsibility; a future proposal may add runtime validation of SASL plugin placement. **Response builder methods** — `respondWith()`, `respondWithError()`, and `respondWithoutReply()` — follow a fluent builder pattern consistent with the `Filter` API. @@ -319,7 +319,7 @@ Routers never need to manage correlation IDs themselves. Having a dedicated builder method rather than a nullable response makes the fire-and-forget case explicit. The runtime can log a warning if a router uses `respondWith()` for a request that has no response, or `respondWithoutReply()` for a request that expects one. -**Error handling.** If the `CompletionStage` completes exceptionally, the runtime treats this as an unrecoverable error and closes the client connection. +**Error handling.** If the `CompletionStage` completes exceptionally, or if the implementation throws an unchecked exception, the runtime treats this as an unrecoverable error and closes the client connection. Routers should generally handle errors within the terms of the Kafka protocol, using `respondWithError()` to generate error responses with appropriate Kafka error codes. Throwing an exception or returning an exceptionally-completed stage should be avoided where possible. From 12a0ac223495a8a5c0c9b234f88c7910c580f02c Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 17 Jun 2026 00:35:19 +0000 Subject: [PATCH 27/30] docs: describe per-connection TopologyServiceImpl threading model Update cache scope and request sending sections to reflect that TopologyServiceImpl is per-connection (wrapping a shared TopologyCache), eliminating the RequestSender race condition. Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 3ec5c1ad..423461c4 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -392,9 +392,10 @@ Only METADATA responses populate partition and broker data. DESCRIBE_CLUSTER is intentionally excluded: it contains brokers and racks but no topic/partition/leader information, and mixing data from different response types risks inconsistency. **Cache scope.** -The cache is shared per router level (not per connection), backed by `ConcurrentHashMap`. +The `TopologyCache` is shared per router level (not per connection), backed by `ConcurrentHashMap`. All connections through the same router level share the same topology view. This is efficient (no per-connection duplication) and necessary for correctness: a leader discovered by connection A must be usable by connection B (see _Shared node address map_ in the Runtime section). +Each connection gets its own `TopologyServiceImpl` wrapping the shared cache; this per-connection instance holds the `RequestSender` used for discovery requests (see _TopologyService request sending_ in the Runtime section). **Authorization and cache scope.** The cache is not scoped by authenticated subject. @@ -621,12 +622,14 @@ The shared address map is populated from two sources: #### TopologyService request sending -The internal `TopologyServiceImpl` is shared per router level (created in the internal `RouterChainFactory`). -Methods that may send requests (`ensureLeadersCached`, `discoverCoordinator`, `topicNames`) use a `RequestSender` functional interface, bound per-connection via `TopologyServiceImpl.bindRequestSender()`. +The `TopologyCache` is shared per router level (created in the internal `RouterChainFactory`), but each connection gets its own `TopologyServiceImpl` instance wrapping the shared cache. +`TopologyServiceImpl` is created in `RouterChainFactory.createRouter()` alongside the `Router` instance and passed to both the `Router` (via `RouterFactoryContext.topologyService()`) and the `RoutingDecisionHandler`. + +Methods that may send requests (`leaders`, `coordinators`) use a `RequestSender` functional interface, bound per-connection via `TopologyServiceImpl.bindRequestSender()`. The binding happens in `RoutingDecisionHandler.dispatchDynamically()` before each `Router.onRequest()` call — the sender lambda captures the per-request `RouterContextImpl`. -The `RequestSender` field is volatile. -This is safe because all calls to `TopologyService` methods happen on the event loop thread during `onRequest` processing, and `bindRequestSender` is called on the same thread before `onRequest`. +Because each `TopologyServiceImpl` is per-connection, the `RequestSender` field is a plain (non-volatile) field — all access happens on the single event loop thread assigned to the connection's channel. +No synchronisation is needed. #### Node ID mapping implementation From 9fada5feb8b923ec558e77651eff14e86df51b53 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 17 Jun 2026 00:44:23 +0000 Subject: [PATCH 28/30] docs: document SASL plugin placement rules Clarify that SaslInitiator on per-route filter chains authenticates the proxy to the upstream cluster and does not affect authenticatedSubject(). Document the four placement rules that ensure composability and avoid pathological configurations. Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 16 ++++++++++++---- 1 file changed, 12 insertions(+), 4 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 423461c4..f4627f26 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -254,10 +254,18 @@ For shared (many-to-one) mappings — where a single virtual node may serve brok **`sessionId()`** and **`authenticatedSubject()`** provide observability and identity context. `sessionId()` returns a string that uniquely identifies the connection with the Kafka client. It will have the same value for all invocations of `onRequest()` which happen for that client connection, both for the same router over time, and different routers in a topology. -`authenticatedSubject()` returns the client's `Subject` established by mTLS or SASL processing on the VC filter chain (e.g. a SASL termination filter). +`authenticatedSubject()` returns the client's `Subject` established by mTLS or SASL processing on the VC filter chain (e.g. a `SaslTerminator` or `SaslInspector` filter). If no authentication has occurred, the subject will be anonymous. -Correct placement of SASL plugins in the topology is the administrator's responsibility; -a future proposal may add runtime validation of SASL plugin placement. +Note that `SaslInitiator` filters on per-route filter chains authenticate the _proxy_ to the _upstream cluster_ — they do not affect the value returned by `authenticatedSubject()`. + +To ensure composability and avoid pathological configurations, SASL plugins should follow these placement rules: + +* `SaslTerminator` and `SaslInspector` should only appear on the VC filter chain (not on per-route filter chains). +* There should be at most one `SaslTerminator` or `SaslInspector` per VC. +* `SaslInitiator` should only appear on routes whose target is a cluster (not a router). +* There should be at most one `SaslInitiator` per such route. + +Enforcement of these rules is currently the administrator's responsibility; a future proposal may add runtime validation (see the _plugin constraint_ concept sketched in [this gist](https://gist.github.com/tombentley/31971d5fba024cc32a6769bd0d799b52)). **Response builder methods** — `respondWith()`, `respondWithError()`, and `respondWithoutReply()` — follow a fluent builder pattern consistent with the `Filter` API. Rather than constructing `RouterResult` subtypes directly, routers use the builder to construct responses: @@ -836,6 +844,6 @@ This section summarises the key design choices made in this proposal, for ease o * **Routes are scoped to their parent router**, not top-level entities. Route names must be unique within the containing router. This avoids a fourth top-level definition list and reflects the fact that a route's meaning depends on its router. * **Explicit route `id`** decouples the virtual node ID mapping from route ordering. The `id` is an integer in `[0, S)` used in the formula `V = id + S × t`. Making it explicit means reordering routes in the YAML does not change the mapping, avoiding accidental virtual node ID shifts that would invalidate connected clients' cached metadata. * **Routes generalise filter chains.** A route may carry its own filter list in addition to a target, allowing different filter chains for different paths through the graph. -* **Route-level SASL** (e.g. SASL initiator for upstream authentication) is achieved via per-route filters rather than a dedicated route-level property. +* **Route-level SASL** (e.g. SASL initiator for upstream authentication) is achieved via per-route filters rather than a dedicated route-level property. `SaslInitiator` filters on per-route chains authenticate the proxy to the upstream cluster; they do not affect `authenticatedSubject()`. Placement rules for SASL plugins are described in the `RouterContext` section above. * **Definition names** are global within their type (filters, routers, clusters each have their own namespace) but not across types. * **Version capping is the router's responsibility**, not the runtime's. The runtime provides the baseline intersection of proxy and backend version ranges; routers may further constrain as needed. From 8f4d36535d3cd9d804028fe2d236cc32d39da2c0 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 17 Jun 2026 08:50:11 +0000 Subject: [PATCH 29/30] docs: make topicNames() route-scoped in routing API proposal Per-route filter chains can transform topic names differently, so the same cluster-level topic ID can map to different router-visible names on different routes. When allowSharedClusterTargets is enabled, omitting the route parameter causes the cache to return whichever name was written last. Change TopologyService.topicNames(Set) to topicNames(String route, Set) and update invalidation semantics to clear topic name entries on invalidateRoute(). Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index f4627f26..60a535c3 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -342,7 +342,7 @@ interface TopologyService { // Discovery (async, may send requests, return self-contained results) CompletionStage leaders(Map> topicsByRoute); CompletionStage coordinators(String route, byte keyType, Set keys); - CompletionStage> topicNames(Set topicIds); + CompletionStage> topicNames(String route, Set topicIds); // Supplementary lookups (sync, best-effort from cache) Optional partitionInfo(String topicName, int partitionIndex); @@ -368,15 +368,16 @@ They may send requests internally (METADATA for leaders and topic names, FIND_CO The results are valid immediately upon completion and do not require further cache queries. `leaders()` batches all uncached topics into one METADATA request per route and returns a `PartitionLeaders` snapshot. `coordinators()` sends METADATA (if needed) then FIND_COORDINATOR and returns a `Coordinators` snapshot; it supports batched lookup matching the FIND_COORDINATOR v4+ protocol. -`topicNames()` resolves topic IDs to names, batching cache misses. +`topicNames()` resolves topic IDs to names for a given route, batching cache misses. +The route parameter is required because per-route filter chains can transform topic names differently — the same cluster-level topic ID can map to different router-visible names on different routes. These methods use a `RequestSender` bound per-connection by the runtime (see _Runtime_ below). **Supplementary lookups** — `partitionInfo()`, `brokerInfo()` — query the cache synchronously and return immediately. These are for use cases like follower-fetch / AZ-aware routing where the router needs replica or rack information beyond what `PartitionLeaders` provides. If they return empty, the router can fall back to the leader from `PartitionLeaders`. -**Invalidation** — `invalidateRoute(route)` performs coarse invalidation: clears all partition info (leaders, replicas, ISR), coordinators, and broker info for a route. -Topic ID→name mappings are _not_ cleared (they are stable within a cluster — a topic ID always maps to the same name). +**Invalidation** — `invalidateRoute(route)` performs coarse invalidation: clears all partition info (leaders, replicas, ISR), coordinators, broker info, and topic ID→name mappings for a route. +Topic ID→name mappings are route-scoped because per-route filter chains can present different names for the same underlying cluster topic. Over-invalidation is acceptable because the cache is repopulated cheaply from the client's next METADATA request. **Why discovery methods return result objects, not void.** @@ -834,10 +835,10 @@ This section summarises the key design choices made in this proposal, for ease o * **`TopologyService` is opt-in** via `RouterFactoryContext.topologyService()`. The runtime creates the underlying cache lazily on first call. Simple routers (subject-based, client-ID-based) that never call it pay no cost — no cache, no topology-related processing. This avoids a one-size-fits-all design where all routers pay for topology caching. * **Side-effect cache population** — the topology cache is populated from METADATA and FIND_COORDINATOR responses in `RoutingDecisionHandler.write()`, after node ID translation, before the router's `CompletionStage` completes. Discovery methods (`leaders()`, `coordinators()`) return self-contained snapshot objects (`PartitionLeaders`, `Coordinators`) that read from the now-warm cache. The snapshots are safe to use across async callback boundaries. * **Discovery methods return snapshots, not void** — an earlier design had `ensureLeadersCached()` returning `CompletionStage` with separate synchronous `leaderOf()`/`coordinatorOf()` methods. This was rejected because the synchronous methods are unsafe: the shared cache can be invalidated between the discovery future completing and the synchronous query. Returning a self-contained snapshot eliminates this race. It also makes the three discovery methods symmetric: `leaders()`, `coordinators()`, `topicNames()` all return self-contained results. -* **Coarse invalidation** via `invalidateRoute(route)` rather than fine-grained `invalidateLeader()`, `invalidateCoordinator()`, `invalidateNode()`. A single method clears partition info, coordinators, and broker info for a route (but not topic ID→name mappings, which are stable within a cluster). Over-invalidation is acceptable because the cache is repopulated cheaply from the client's next METADATA request. This avoids an ever-growing set of invalidation methods as more cached entity types are added. +* **Coarse invalidation** via `invalidateRoute(route)` rather than fine-grained `invalidateLeader()`, `invalidateCoordinator()`, `invalidateNode()`. A single method clears partition info, coordinators, broker info, and topic ID→name mappings for a route. Over-invalidation is acceptable because the cache is repopulated cheaply from the client's next METADATA request. This avoids an ever-growing set of invalidation methods as more cached entity types are added. * **Router-driven invalidation** rather than runtime-driven. The runtime would need to deserialise an open-ended and growing set of response types to scan for staleness error codes. Different error codes have different semantics. Routers that don't use the topology cache shouldn't pay the deserialisation cost. * **Shared node address map** — broker address resolution is per-router-level (`ConcurrentHashMap`), not per-connection. Required because the shared topology cache means a leader virtual node ID cached from one connection's METADATA must be resolvable to a broker address on another connection. -* **Topic ID resolution via `TopologyService.topicNames(Set)`** is async and batched, consistent with `leaders()` and `coordinators()`. An earlier design had a synchronous `topicName(Uuid)` on `RouterContext`, preloaded by an internal filter before `onRequest()`. This was removed because it silently swallowed resolution errors (returning null for both "not yet resolved" and "topic deleted") and was inconsistent with the async discovery pattern. Routers that need topic names collect IDs from the request, call `topicNames()`, and pass the resolved map to decomposers. +* **Topic ID resolution via `TopologyService.topicNames(String route, Set)`** is async, batched, and route-scoped, consistent with `leaders()` and `coordinators()`. An earlier design had a synchronous `topicName(Uuid)` on `RouterContext`, preloaded by an internal filter before `onRequest()`. This was removed because it silently swallowed resolution errors (returning null for both "not yet resolved" and "topic deleted") and was inconsistent with the async discovery pattern. Routers that need topic names collect IDs from the request, call `topicNames()`, and pass the resolved map to decomposers. The route parameter is required because per-route filter chains can transform topic names differently: the same cluster-level topic ID can map to different router-visible names on different routes. When `allowSharedClusterTargets()` is in effect, omitting the route would cause the cache to return whichever name was written last, silently corrupting routing decisions. * **Routes must target distinct clusters by default.** The runtime rejects configurations where two routes in the same router resolve (directly or transitively) to the same cluster, because most routers assume each route is a distinct destination. Router factories that handle shared cluster targets (e.g. aliasing the same cluster under different prefixes) call `allowSharedClusterTargets()` during `initialize()` to suppress the check. The check runs in the internal `RouterChainFactory` after factory initialization, not in the internal `RouterGraphValidator`, because it depends on the factory's runtime declaration. * **`routeNames()` in `RouterFactoryContext`** allows router factories to validate at initialization time that route names referenced in their plugin configuration actually exist, providing early feedback on configuration errors. * **`target` as a discriminated union** provides a uniform way to express "what does this connect to" across both virtual clusters and routes, with the type (`cluster` or `router`) made explicit inside the object. From 13d12fb1b933a16a899300f00ed2cdd95c7705b6 Mon Sep 17 00:00:00 2001 From: Tom Bentley Date: Wed, 17 Jun 2026 22:29:11 +0000 Subject: [PATCH 30/30] docs: acknowledge route addition/removal invalidates virtual node IDs MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The dedicated mapping formula V = id + S × t uses the route count S as a multiplier. Adding or removing routes changes S, shifting all virtual node IDs for existing routes. Acknowledge this explicitly in both the mapping strategy section and operational considerations, noting that it requires draining all connections and reconfiguring all proxy instances together. Assisted-by: Claude Opus 4.6 Signed-off-by: Tom Bentley --- proposals/070-routing-api.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/proposals/070-routing-api.md b/proposals/070-routing-api.md index 60a535c3..c3f1ca92 100644 --- a/proposals/070-routing-api.md +++ b/proposals/070-routing-api.md @@ -672,6 +672,11 @@ Likewise, adding or removing proxy instances has no effect on the mapping. Because the route's `id` is explicit rather than derived from position, reordering routes in the configuration does not change the mapping. This stability is important because Kafka clients cache broker metadata and will use previously-seen node IDs in subsequent requests. +However, _adding or removing_ a route changes `S`, which shifts every virtual node ID for every existing route (for any `t > 0`). +This is an accepted consequence of the formula: virtual cluster configuration changes already drain all client connections before applying the new configuration, so clients reconnect and receive fresh metadata. +All proxy instances presenting the same virtual cluster must be reconfigured together (i.e. the configuration change must be applied to all instances before any drained clients reconnect), so that clients see a consistent set of virtual node IDs. +A future formula that avoids the `S`-dependency (e.g. a fixed block size per route) could enable more surgical configuration changes, but is not required for the current dedicated mapping. + For single-route configurations, an identity mapping (a special case of dedicated mapping) is used: the virtual ID equals the target ID, with zero overhead. ##### Per-router scoping and nested dispatch @@ -717,6 +722,8 @@ If roles span routes, it behaves like a shared mapping. All proxy instances presenting the same virtual cluster to clients must use the same `NodeIdMapping`. Changing the mapping strategy (e.g. from dedicated to shared) is expected to be a disruptive operation — not a zero-downtime change — because connected clients will hold stale virtual node IDs from the previous mapping. +Similarly, adding or removing routes changes `S` in the dedicated mapping formula, which invalidates existing virtual node IDs. +This requires draining all client connections and reconfiguring all proxy instances together before clients reconnect. `NodeIdMapping` is a `sealed` interface — the runtime controls which implementations exist. `Router` authors never implement or interact with this interface directly.