Installation
Multus can be deployed by simply applying thethick
DaemonSet
with kubectl
.
DaemonSet
and a CRD: NetworkAttachmentDefinition
.
This can be used to specify your network configuration.
Configuration
Patching the DaemonSet
For Multus to properly work with Talos a change need to be made to the DaemonSet
.
Instead of of mounting the volume called host-run-netns
on /run/netns
it has to be mounted on /var/run/netns
.
Edit the DaemonSet
and change the volume host-run-netns
from /run/netns
to /var/run/netns
.
install-multus-binary
container that can be lead to race problems after a node reboot.
To prevent this issue, it is necessary to patch this container.
Set the following command to the install-multus-binary
container.
Creating your NetworkAttachmentDefinition
The NetworkAttachmentDefinition
configuration is used to define your bridge where your second pod interface needs to be attached to.
macvlan
is used as a bridge type.
There are 3 types of bridges: bridge
, macvlan
and ipvlan
:
-
bridge
is a way to connect two Ethernet segments together in a protocol-independent way. Packets are forwarded based on Ethernet address, rather than IP address (like a router). Since forwarding is done at Layer 2, all protocols can go transparently through a bridge. In terms of containers or virtual machines, a bridge can also be used to connect the virtual interfaces of each container/VM to the host network, allowing them to communicate. -
macvlan
is a driver that makes it possible to create virtual network interfaces that appear as distinct physical devices each with unique MAC addresses. The underlying interface can route traffic to each of these virtual interfaces separately, as if they were separate physical devices. This means that each macvlan interface can have its own IP subnet and routing. Macvlan interfaces are ideal for situations where containers or virtual machines require the same network access as the host system. -
ipvlan
is similar tomacvlan
, with the key difference being that ipvlan shares the parent’s MAC address, which requires less configuration from the networking equipment. This makes deployments simpler in certain situations where MAC address control or limits are in place. It offers two operational modes: L2 mode (the default) where it behaves similarly to a MACVLAN, and L3 mode for routing based traffic isolation (rather than bridged).
bridge
interface you must also configure a bridge on your Talos nodes.
That can be done by updating Talos Linux machine configuration:
Attaching the NetworkAttachmentDefinition
to your Pod
or Deployment
After the NetworkAttachmentDefinition
is configured, you can attach that interface to your your Deployment
or Pod
.
In this example we use a pod:
Notes on using KubeVirt in combination with Multus
If you would like to use KubeVirt and expose your virtual machine to the outside world with Multus, make sure to configure abridge
instead of macvlan
or ipvlan
, because that doesn’t work, according to the KubeVirt Documentation.
Invalid CNIs for secondary networks The following list of CNIs is known not to work for bridge interfaces - which are most common for secondary interfaces.The reason is similar: the bridge interface type moves the pod interface MAC address to the VM, leaving the pod interface with a different address. The aforementioned CNIs require the pod interface to have the original MAC address.
- macvlan
- ipvlan
Notes on using Cilium in combination with Multus
CNI reference plugins
Cilium does not ship the CNI reference plugins, which most multus setups are expecting (e.g. macvlan). This can be addressed by extending the daemonset with an additional init-container, setting them up, e.g. using the following kustomize strategic-merge patch:Updates on the install-cni
image.
In earlier Talos versions, an image called siderolabs/install-cni
was required to provide the standard CNI reference plugins.
This image has since been deprecated.
With the newer Talos versions, the essential CNI plugins are already bundled with Talos Linux itself.
This means you generally do not need to use siderolabs/install-cni
anymore.
Multus and most CNIs should work with the plugins shipped in Talos out of the box.
If you need additional or custom CNI plugins that are not included, you can:
- Use the official upstream builds, or
- Build your own plugin image and deploy it with a DaemonSet
siderolabs/install-cni
in this guide remain for historical context and compatibility examples, but new deployments should rely on the bundled plugins in Talos unless a custom requirement exists.
Exclusive CNI
By default, Cilium is an exclusive CNI, meaning it removes other CNI configuration files. However, when using Multus, this behavior needs to be disabled. To do so, set the Helm variablecni.exclusive=false
.
For more information, refer to the Cilium documentation.