Add on: Multus

Multus is a CNI manager which enables attaching multiple network interfaces to pods in Kubernetes (details here).

The install is transparent and the current CNI plugin is carried forward as the default in Multus. With Multus enabled, users can easily add multiple interfaces to pods using varying CNI plugins (like bridge, macvlan, ipvlan, ptp, etc.).

This is handy for small local cases where you want to attach to local networks/vlans or in much more complex setups where multiple more fully featured CNI plugins can be used simultaneously.

Enable the Multus add-on by running the command:

microk8s enable multus

You can confirm it’s installed by verifying the daemonset is running:

microk8s kubectl get pods -n kube-system --selector=app=multus

Which should return output similar to the following:

NAME                         READY   STATUS    RESTARTS   AGE
kube-multus-ds-amd64-gtxq2   1/1     Running   0          2m36s

Once enabled you can create network attachment definitions for various networks you want your pods to connect to. Here is an example that creates two networks that attach to Linux bridges (br100 & br200) on the host machine:

---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: home-network
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "home-network",
    "type": "bridge",
    "bridge": "br100",
    "isDefaultGateway": false,
    "forceAddress": false,
    "ipMasq": false,
    "hairpinMode": false,
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.1.0/24",
      "rangeStart": "192.168.1.201",
      "rangeEnd": "192.168.1.250",
      "routes": [
        { "dst": "0.0.0.0/0" }
      ],
      "gateway": "192.168.1.1"
    }
  }'
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: work-network
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "work-network",
    "type": "bridge",
    "bridge": "br200",
    "isDefaultGateway": false,
    "forceAddress": false,
    "ipMasq": false,
    "hairpinMode": false,
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.2.0/24",
      "rangeStart": "192.168.2.201",
      "rangeEnd": "192.168.2.250",
      "routes": [
        { "dst": "0.0.0.0/0" }
      ],
      "gateway": "192.168.2.1"
    }
  }'

Then you can create pods that attach to those networks using annotations. Here is an example of an alpine image that connects to the two above networks:

---
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: multus-alpine
  name: multus-alpine
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/networks: '[
      {
         "name" : "home-network",
         "interface": "eth1",
         "ips": ["192.168.1.205"]
      },
      {
         "name" : "work-network",
         "interface": "eth2",
         "ips": ["192.168.2.210"]
      }
    ]'
spec:
  containers:
    - name: multus-alpine
      image: alpine:latest
      command: ["sh"]
      args: ["-c", "while [ true ]; do ifconfig; sleep 3; done"]
  restartPolicy: Always

This example uses an assigned IP but if you leave it out one will be assigned from the network range.

Should you need to disable the plugin you can do so by running:

microk8s disable multus

More details can be found at the Multus page. In particular in the how-to-use section of the repository docs:
k8snetworkplumbing multus-cni repository

Multus CNI enables attaching multiple network interfaces to pods in Kubernetes (details here). The install is transparent and the current CNI plugin is carried forward as the default in Multus. With Multus installed users can easily add multiple interfaces to pods using varying CNI plugins (like bridge, macvlan, ipvlan, ptp, etc.). This is handy for small local cases where you want to attach to local networks/vlans or in much more complex setups where multiple more fully featured CNI plugins can be used simultaneously.

Enable the multus add-on by running the command:

microk8s enable multus

You can confirm it’s installed by verifying the daemonset is running:

microk8s kubectl get pods -n kube-system --selector=app=multus

Which should return output similar to the following:

NAME                         READY   STATUS    RESTARTS   AGE
kube-multus-ds-amd64-gtxq2   1/1     Running   0          2m36s

Once enabled you can create network attachment definitions for various networks you want your pods to connect to. Here is an example that creates two networks that attach to linux bridges (br100 & br200) on the host machine:

---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: home-network
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "home-network",
    "type": "bridge",
    "bridge": "br100",
    "isDefaultGateway": false,
    "forceAddress": false,
    "ipMasq": false,
    "hairpinMode": false,
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.1.0/24",
      "rangeStart": "192.168.1.201",
      "rangeEnd": "192.168.1.250",
      "routes": [
        { "dst": "0.0.0.0/0" }
      ],
      "gateway": "192.168.1.1"
    }
  }'
---
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: work-network
spec:
  config: '{
    "cniVersion": "0.3.1",
    "name": "work-network",
    "type": "bridge",
    "bridge": "br200",
    "isDefaultGateway": false,
    "forceAddress": false,
    "ipMasq": false,
    "hairpinMode": false,
    "ipam": {
      "type": "host-local",
      "subnet": "192.168.2.0/24",
      "rangeStart": "192.168.2.201",
      "rangeEnd": "192.168.2.250",
      "routes": [
        { "dst": "0.0.0.0/0" }
      ],
      "gateway": "192.168.2.1"
    }
  }'

Then you can create pods that attach to those networks using annotations. Here is an example of an alpine image that connects to the two above networks:

---
apiVersion: v1
kind: Pod
metadata:
  labels:
    app: multus-alpine
  name: multus-alpine
  namespace: default
  annotations:
    k8s.v1.cni.cncf.io/networks: '[
      {
         "name" : "home-network",
         "interface": "eth1",
         "ips": ["192.168.1.205"]
      },
      {
         "name" : "work-network",
         "interface": "eth2",
         "ips": ["192.168.2.210"]
      }
    ]'
spec:
  containers:
    - name: multus-alpine
      image: alpine:latest
      command: ["sh"]
      args: ["-c", "while [ true ]; do ifconfig; sleep 3; done"]
  restartPolicy: Always

This example uses an assigned IP but if you leave it out one will be assigned from the network range.


Should you need to disable the plugin you can do so by running:

microk8s disable multus

More details can be found at the Multus page. In particular here:
Network Attachment Definitions
Network Annotations

1 Like

thanks @apnar, I’ll get this incorporated into the doc

Hello, I noticed the example of the web page is wrong. It has two NetworkAttachmentDefinitions with home-network.

apiVersion: “k8s.cni.cncf.io/v1
kind: NetworkAttachmentDefinition
metadata:
name: home-network <-----Change to ‘work-network’
spec:
config: ‘{
“cniVersion”: “0.3.1”,
“name”: “work-network”,
“type”: “bridge”,
“bridge”: “br200”,
“isDefaultGateway”: false,
“forceAddress”: false,
“ipMasq”: false,
“hairpinMode”: false,
“ipam”: {
“type”: “host-local”,
“subnet”: “192.168.2.0/24”,
“rangeStart”: “192.168.2.201”,
“rangeEnd”: “192.168.2.250”,
“routes”: [
{ “dst”: “0.0.0.0/0” }
],
“gateway”: “192.168.2.1”
}
}’

Thanks @Mikez, good catch. I can’t edit the above posts to correct it but maybe @evilnick can.

thanks @MikeZ , updated it now!

The links at the bottom of the page are also invalid and result in a 404

also fixed

Looks like the NetworkAttachmentDefinition name is wrong again

1 Like

In the microk8s multus EXAMPLE created Error from server (AlreadyExists): error when creating “/tmp/nad.yml”: network-attachment-definitions.k8s.cni.cncf.io “home-network” already exists), both net-attach-def (s) have the same name home-network, please fix to prevent error.

Error:

networkattachmentdefinition.k8s.cni.cncf.io/home-network created
Error from server (AlreadyExists): error when creating "/tmp/nad.yml": network-attachment-definitions.k8s.cni.cncf.io "home-network" already exists

I’m not sure when it happened, but it seems multus is now in the stable channel so the disclaimer about edge is no longer needed. I was just able to enable and use multus with v1.20.6.

1 Like