You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Enhance NetworkInterface Address and State handling #1541
Support reporting the allocated IP addresses, prefixes, and interface state via IRI.
Make the machinepoollet the controller of the network interface associated with its machine pool, including writing back the status from the IRI response.
Allow creation of network interfaces without IP addresses or IP families.
Motivation
Opening up the contract and handing control over the network interface to the machinepoollet allows creating novel network implementations that don't assume a top-down approach for IP addresses. It also lifts the requirement of implementing a dedicated control-plane for managing the status of the network interface like it is required today.
Looking in more detail at the current IRI API surface and how it's used I believe we should not adjust anything except the Machine message type. The pollet anyway uses that as its source of truth and with asynchronous providers there is nothing to return on the initial attach call anyway. The question then becomes how we keep IRI relatively simple (the network interface status always lives inside the machine status) while also allowing the poollet to report the effective values at the right place (machine status vs. network interface status).
In that case, the change is almost trivial, we just add the new fields to the NetworkInterfaceStatus and extend the NetworkInterfaceState so that it can cover the lifecycle of both states, the one inside the machine and the one in the network interface:
For the poollet the state becomes a threshold function:
pending: both pending.
ready: the network interface is ready and it's data (IPs, prefixes, virtual IP) are final, machine side stays pending.
error: network interface creation failed which is mirrored into the network interface state, machine side stays pending
attached: has been attached to the VM, network interface automatically becomes ready, machine side becomes attached.
If a provider does not report the additional data poollet will initially not fill it in, leaving whatever component currently manages the NIC state to do that. For the current ironcore-net use-case the NIC is most likely already set to ready when the poollet gets word of it, that should also not be an issue. The poollet should initially ignore if someone else populates the ip / prefix / vip fields, but eventually we might want to give it full ownership and have it remove anything it isn't aware of.
As for breaking vs. non-breaking change: the change in itself is compatible thanks to protobuf. The only thing to be aware of is that a provider reporting a new state to an old poollet would trigger an error as it doesn't know how to handle that. Since poollet and provider are anyway usually deployed as one this should not be an issue. Pairing a newer poollet with an old provider should also work without any issues.
On the Kubernetes API side we only need to mark the IP families and IP addresses field as optional to allow creation without those.
@adracus and I discussed the API changes and agreed to the following adjustments:
Drop the virtual IP from the status. Virtual IPs are usually attached in some way and not an inherent property of the network interface. We currently don't see in which case a network plugin would report a VIP. If that ever comes up, it's cheap to add at a later point.
Add the network interface metadata to the IRI network interface. This makes it clear that the poollet is operating on the actual network interface and allows for future logic to group networks based on namespaces or other metadata, should that be desired.
With the addition of metadata we get another interesting behavior: the only way for the machinepoollet to update a network interface is through re-attaching, there is no update RPC it could call. If you are managing your resources via Flux or some other component does bookkeeping in annotations every change would cause your NIC to be re-attached which seems wrong. We have two options:
Never update metadata on its own, only pass the metadata at attach.
Introduce a new RPC to update the NIC without re-attaching it.
While testing a rough implementation to see if this fits our use-case for a network plugin which propagates the IP address up it became clear that running two networks in parallel is dangerous. With apinetlet assuming responsibility over all network interfaces it reverts the status to pending after the machinepoollet sets it to ready. A workaround is to annotate it with common.ironcore.dev/managed-by, maybe that will be the transition plan then, have the poollet annotate every network interface it takes over that way to have a smooth transition.
If I'm understanding correctly the new state transition would be PENDING -> READY -> ATTACHED. I guess that should be documented in the protobuf code. Potentially, it would make sense to swap READY and ATTACHED in the declaration. Then, the normal transition flow would be in the declaration. From protobuf POV that should not be a real change if the IDs are retained.
And on the naming: Is the READY -> ATTACHED transition obvious for everyone? If not, renaming ATTACHED to READ_AND_ATTACHED would make it explicit also for code using the enum.
Btw, do we need to care about backward compatibility already? If not, can't we actually renumber the enum variants still? Or would breaking changes require a version bump? In that case, it could be postponed.
If I'm understanding correctly the new state transition would be PENDING -> READY -> ATTACHED. I guess that should be documented in the protobuf code. Potentially, it would make sense to swap READY and ATTACHED in the declaration. Then, the normal transition flow would be in the declaration. From protobuf POV that should not be a real change if the IDs are retained.
Yes, your understanding is correct. The code block in my comment was primarily there to make clear what would change on the API level, the PR will contain some doc comments and I can also re-order the lines to match the actual flow.
And on the naming: Is the READY -> ATTACHED transition obvious for everyone? If not, renaming ATTACHED to READ_AND_ATTACHED would make it explicit also for code using the enum.
I'd prefer to keep the distinct states with their own names and document the semantics accordingly. State transitions are something we commonly deal with and later states usually don't include all of the states that had to be transitioned before them.
Btw, do we need to care about backward compatibility already? If not, can't we actually renumber the enum variants still? Or would breaking changes require a version bump? In that case, it could be postponed.
IMO we can still renumber, just make sure we add a breaking notice to our release. For a consumer, this means they have to bump their IRI-version in accordance with their implementation (e.g. provider-libvirt).
It has turned out that this is a much larger change than I originally anticipated. With the poollet reporting the NIC state we are getting into a race with the apinetlet from ironcore-net. To resolve this we need to update it to no longer manage the NIC itself and instead rely on the machinepoollet. For this to work we also need to re-work the apinet code-paths in libvirt-provider to report the status of the NICs like I've done it for the new plugin in my PoC.
I'm currently working to prepare a first PR for IronCore itself to get some early feedback on it and will prepare the necessary changes in the other components as well to have a complete picture.
I've started unraveling all the pieces necessary to get something ready for a PR. Due to the brokering change I first have to fix some outdated imports and bump some dependencies, starting with the provider-utils:
And now here's the draft PR for the ironcore change itself. I still have to figure out some of the details but the change overall is going into the right direction, so it's something you can already look at and raise concerns on: #1578
This ensures we do not trigger the race between the machinepoollet and apinetlet and that the provider only starts using the new APIs once the machinepoollet is ready to consume them (this can probably also be done in one step).
Now I also have a draft PR for libvirt-provider, that one needs to wait until we can consume the new ironcore version to use the new proto APIs: ironcore-dev/libvirt-provider#791
Activity
maxmoehl commentedon Aug 21, 2026
Looking in more detail at the current IRI API surface and how it's used I believe we should not adjust anything except the
Machinemessage type. The pollet anyway uses that as its source of truth and with asynchronous providers there is nothing to return on the initial attach call anyway. The question then becomes how we keep IRI relatively simple (the network interface status always lives inside the machine status) while also allowing the poollet to report the effective values at the right place (machine status vs. network interface status).In that case, the change is almost trivial, we just add the new fields to the
NetworkInterfaceStatusand extend theNetworkInterfaceStateso that it can cover the lifecycle of both states, the one inside the machine and the one in the network interface:For the poollet the state becomes a threshold function:
pending: both pending.ready: the network interface is ready and it's data (IPs, prefixes, virtual IP) are final, machine side stays pending.error: network interface creation failed which is mirrored into the network interface state, machine side stays pendingattached: has been attached to the VM, network interface automatically becomes ready, machine side becomes attached.If a provider does not report the additional data poollet will initially not fill it in, leaving whatever component currently manages the NIC state to do that. For the current ironcore-net use-case the NIC is most likely already set to ready when the poollet gets word of it, that should also not be an issue. The poollet should initially ignore if someone else populates the ip / prefix / vip fields, but eventually we might want to give it full ownership and have it remove anything it isn't aware of.
As for breaking vs. non-breaking change: the change in itself is compatible thanks to protobuf. The only thing to be aware of is that a provider reporting a new state to an old poollet would trigger an error as it doesn't know how to handle that. Since poollet and provider are anyway usually deployed as one this should not be an issue. Pairing a newer poollet with an old provider should also work without any issues.
On the Kubernetes API side we only need to mark the IP families and IP addresses field as optional to allow creation without those.
maxmoehl commentedon Aug 21, 2026
@adracus and I discussed the API changes and agreed to the following adjustments:
The updated patch is:
maxmoehl commentedon Aug 21, 2026
With the addition of metadata we get another interesting behavior: the only way for the machinepoollet to update a network interface is through re-attaching, there is no update RPC it could call. If you are managing your resources via Flux or some other component does bookkeeping in annotations every change would cause your NIC to be re-attached which seems wrong. We have two options:
maxmoehl commentedon Aug 21, 2026
While testing a rough implementation to see if this fits our use-case for a network plugin which propagates the IP address up it became clear that running two networks in parallel is dangerous. With apinetlet assuming responsibility over all network interfaces it reverts the status to pending after the machinepoollet sets it to ready. A workaround is to annotate it with
common.ironcore.dev/managed-by, maybe that will be the transition plan then, have the poollet annotate every network interface it takes over that way to have a smooth transition.johamster commentedon Aug 23, 2026
On the added states to
NetworkInterfaceState, also copied below:enum NetworkInterfaceState { NETWORK_INTERFACE_PENDING = 0; NETWORK_INTERFACE_ATTACHED = 1; + NETWORK_INTERFACE_READY = 2; + NETWORK_INTERFACE_ERROR = 3; }If I'm understanding correctly the new state transition would be
PENDING -> READY -> ATTACHED. I guess that should be documented in the protobuf code. Potentially, it would make sense to swapREADYandATTACHEDin the declaration. Then, the normal transition flow would be in the declaration. From protobuf POV that should not be a real change if the IDs are retained.And on the naming: Is the
READY -> ATTACHEDtransition obvious for everyone? If not, renamingATTACHEDtoREAD_AND_ATTACHEDwould make it explicit also for code using the enum.Btw, do we need to care about backward compatibility already? If not, can't we actually renumber the enum variants still? Or would breaking changes require a version bump? In that case, it could be postponed.
maxmoehl commentedon Aug 24, 2026
Yes, your understanding is correct. The code block in my comment was primarily there to make clear what would change on the API level, the PR will contain some doc comments and I can also re-order the lines to match the actual flow.
I'd prefer to keep the distinct states with their own names and document the semantics accordingly. State transitions are something we commonly deal with and later states usually don't include all of the states that had to be transitioned before them.
Renumbering or not is something I defer to @afritzler and @adracus.
adracus commentedon Aug 24, 2026
IMO we can still renumber, just make sure we add a breaking notice to our release. For a consumer, this means they have to bump their IRI-version in accordance with their implementation (e.g.
provider-libvirt).github-actions commentedon Sep 24, 2026
This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.
maxmoehl commentedon Sep 28, 2026
It has turned out that this is a much larger change than I originally anticipated. With the poollet reporting the NIC state we are getting into a race with the apinetlet from ironcore-net. To resolve this we need to update it to no longer manage the NIC itself and instead rely on the machinepoollet. For this to work we also need to re-work the apinet code-paths in libvirt-provider to report the status of the NICs like I've done it for the new plugin in my PoC.
I'm currently working to prepare a first PR for IronCore itself to get some early feedback on it and will prepare the necessary changes in the other components as well to have a complete picture.
maxmoehl commentedon Sep 30, 2026
I've started unraveling all the pieces necessary to get something ready for a PR. Due to the brokering change I first have to fix some outdated imports and bump some dependencies, starting with the provider-utils:
Now I should be able to prepare the bump to ironcore 0.7.0 in libvirt-provider to then be able to cleanly rebase my other changes on top of it:
maxmoehl commentedon Sep 30, 2026
I also have minor changes that kept biting me:
maxmoehl commentedon Sep 30, 2026
And now here's the draft PR for the ironcore change itself. I still have to figure out some of the details but the change overall is going into the right direction, so it's something you can already look at and raise concerns on: #1578
maxmoehl commentedon Oct 1, 2026
And here is the corresponding change for ironcore-net: ironcore-dev/ironcore-net#558
The rollout order would be:
This ensures we do not trigger the race between the machinepoollet and apinetlet and that the provider only starts using the new APIs once the machinepoollet is ready to consume them (this can probably also be done in one step).
maxmoehl commentedon Oct 2, 2026
Now I also have a draft PR for libvirt-provider, that one needs to wait until we can consume the new ironcore version to use the new proto APIs: ironcore-dev/libvirt-provider#791
maxmoehl commentedon Oct 5, 2026
We have now merged all the pull requests. I will keep this issue open a bit longer until we have validated it in our environment.
maxmoehl commentedon Oct 6, 2026
We validated the change in our environment and it seems to work as intended!