Kubeadm KubeletConfiguration Patch - Only on workers

Cluster information:

Kubernetes version:
Installation method: kubeadm
Host OS: Ubuntu 20.04
CNI and version: Calico 3.24.0
CRI and version: containerd 1.6.12

I am trying to bring up a kubeadm cluster with a specific kubelet patch to be applied to only worker nodes. So the basic concept is:

  • bring up the control-plane node
  • apply base kubelet config to all (control or worker)
  • apply supplement (patch) to only worker nodes

Currently, the worker node joins without an issue but the patch is not applied. Not sure if the config file is incorrect.

Process (on control plane):

  1. Run sudo mkdir -p /etc/kubernetes/patches
  2. Copy /etc/kubernetes/patches/kubeletconfiguration0+merge.yaml
  3. Copy /etc/kubernetes/clusterconfiguration.yaml
  4. Run kubeadm init --config=/etc/kubernetes/clusterconfiguration.yaml --upload-certs
  5. Copy the output/cmd for ‘adding worker node to cluster’

Process (on worker node):

  1. Run the output line from the above command

Expected results (on the worker node):

  • less /var/lib/kubernetes/config.yaml
  • should see line protectKernelDefaults: true

kubeletconfiguration0+merge.yaml

# /etc/kubernetes/patches/kubeletconfiguration0+merge.yaml
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
protectKernelDefaults: true
...

clusterconfiguration.yaml

---
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
  - system:bootstrappers:kubeadm:default-node-token
  token: abcdef.0123456789abcdef
  ttl: 1h0m0s
  usages:
  - signing
  - authentication
kind: InitConfiguration
localAPIEndpoint:
  advertiseAddress: <control-plane host ip>
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
  imagePullPolicy: IfNotPresent
  name: <control-plane host FQDN>
  taints: null
---
apiServer:
  timeoutForControlPlane: 4m0s
  extraArgs:
    enable-admission-plugins: AlwaysPullImages,EventRateLimit,NodeRestriction,SecurityContextDeny
    admission-control-config-file: /etc/kubernetes/admission-control-config.yaml
    profiling: "false"
    request-timeout: "300s"
    audit-policy-file: /etc/kubernetes/audit-policy-global.yaml
    audit-log-path: /var/log/kubernetes/audit/audit.log
    audit-log-maxage: '30'
    audit-log-maxbackup: '10'
    audit-log-maxsize: '100'
    kubelet-certificate-authority: /etc/kubernetes/pki/ca.crt
    tls-cipher-suites: TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384
  extraVolumes:
    - name: admissioncontrolconfig
      hostPath: /etc/kubernetes/admission-control-config.yaml
      mountPath: /etc/kubernetes/admission-control-config.yaml
      pathType: "File"
      readOnly: true
    - name: eventratelimit
      hostPath: /etc/kubernetes/eventratelimit-server-namespace-user.yaml
      mountPath: /etc/kubernetes/eventratelimit-server-namespace-user.yaml
      readOnly: true
      pathType: "File"
    - name: audit
      hostPath: /etc/kubernetes/audit-policy-global.yaml
      mountPath: /etc/kubernetes/audit-policy-global.yaml
      readOnly: true
      pathType: "File"
    - name: audit-log
      hostPath: /var/log/kubernetes/audit/
      mountPath: /var/log/kubernetes/audit/
      readOnly: false
      pathType: "DirectoryOrCreate"
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager:
  extraArgs:
    profiling: "false"
    terminated-pod-gc-threshold: '10'
dns: {}
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.24.2
controlPlaneEndpoint: <control-plane host FQDN>
networking:
  dnsDomain: cluster.local
  serviceSubnet: 172.16.0.0/16
  podSubnet: 192.168.0.0/16
scheduler: {}
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
eventRecordQPS: 10
tlsCipherSuites:
  - TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
  - TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
  - TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
  - TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
  - TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
  - TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
  - TLS_RSA_WITH_AES_256_GCM_SHA384
  - TLS_RSA_WITH_AES_128_GCM_SHA256
...
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration
patches:
  directory: /etc/kubernetes/patches
discovery:
  nodeSelector:
    matchLabels:
      node-role.kubernetes.io/worker: ""
...