Switching Flannel to DirectRouting

General Discussions
1

Cluster information:

Kubernetes version: 1.23.7
Cloud being used: bare-metal
Installation method: kubespray
Host OS: CentOS 7.9
CNI and version: Flannel v0.17.0
CRI and version: docker://20.10.11

Hi, I’d like to dynamically switch Flannel’s backend to DirectRouting:

  • DirectRouting (Boolean): Enable direct routes (like host-gw) when the hosts are on the same subnet. VXLAN will only be used to encapsulate packets to hosts on different subnets. Defaults to false. DirectRouting is not supported on Windows.

I know that “once set, the backend should not be changed at runtime”. However, technically it would remain the VXLAN backend, but with an additional option, so I’m hoping this is possible. Additionally, there are reports of people successfully doing this.

This is (part of) the flannel daemonset:

      initContainers:
      - args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        command:
        - cp
        image: docker.io/flannelcni/flannel:v0.17.0-amd64
        imagePullPolicy: IfNotPresent
        name: install-cni
        resources: {}
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /etc/cni/net.d
          name: cni
        - mountPath: /etc/kube-flannel/
          name: flannel-cfg

      volumes:
      - configMap:
          defaultMode: 420
          name: kube-flannel-cfg
        name: flannel-cfg

I added DirectRouting in the configmap kube-flannel-cfg as (partly) shown below:

data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.233.64.0/18",
      "Backend": {
        "Type": "vxlan",
        "VNI": 1,
        "Port": 8472,
        "DirectRouting": true
      }
    }

I then restarted all pods (including the flannel ones).
Everything seems to be working fine.

I would have the following questions please:

  • Is the above procedure correct and sufficient?
  • What would be the easiest way to verify that DirectRouting has indeed been applied?

EDIT: the following command shows nothing:

sudo tcpdump -i 'flannel.1'

On another cluster where DirectRouting isn’t enabled yet, I can see a lot of traffic on this interface, as expected.
So I assume the change to DirectRouting was successful.