Cluster information:
Kubernetes version:
Installation method: kubeadm
Host OS: Ubuntu 20.04
CNI and version: Calico 3.24.0
CRI and version: containerd 1.6.12
I am trying to bring up a kubeadm cluster with a specific kubelet patch to be applied to only worker nodes. So the basic concept is:
- bring up the control-plane node
- apply base kubelet config to all (control or worker)
- apply supplement (patch) to only worker nodes
Currently, the worker node joins without an issue but the patch is not applied. Not sure if the config file is incorrect.
Process (on control plane):
- Run
sudo mkdir -p /etc/kubernetes/patches
- Copy
/etc/kubernetes/patches/kubeletconfiguration0+merge.yaml
- Copy
/etc/kubernetes/clusterconfiguration.yaml
- Run
kubeadm init --config=/etc/kubernetes/clusterconfiguration.yaml --upload-certs
- Copy the output/cmd for ‘adding worker node to cluster’
Process (on worker node):
- Run the output line from the above command
Expected results (on the worker node):
- less /var/lib/kubernetes/config.yaml
- should see line
protectKernelDefaults: true
kubeletconfiguration0+merge.yaml
# /etc/kubernetes/patches/kubeletconfiguration0+merge.yaml
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
protectKernelDefaults: true
...
clusterconfiguration.yaml
---
apiVersion: kubeadm.k8s.io/v1beta3
bootstrapTokens:
- groups:
- system:bootstrappers:kubeadm:default-node-token
token: abcdef.0123456789abcdef
ttl: 1h0m0s
usages:
- signing
- authentication
kind: InitConfiguration
localAPIEndpoint:
advertiseAddress: <control-plane host ip>
bindPort: 6443
nodeRegistration:
criSocket: unix:///run/containerd/containerd.sock
imagePullPolicy: IfNotPresent
name: <control-plane host FQDN>
taints: null
---
apiServer:
timeoutForControlPlane: 4m0s
extraArgs:
enable-admission-plugins: AlwaysPullImages,EventRateLimit,NodeRestriction,SecurityContextDeny
admission-control-config-file: /etc/kubernetes/admission-control-config.yaml
profiling: "false"
request-timeout: "300s"
audit-policy-file: /etc/kubernetes/audit-policy-global.yaml
audit-log-path: /var/log/kubernetes/audit/audit.log
audit-log-maxage: '30'
audit-log-maxbackup: '10'
audit-log-maxsize: '100'
kubelet-certificate-authority: /etc/kubernetes/pki/ca.crt
tls-cipher-suites: TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256,TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256,TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_RSA_WITH_AES_256_GCM_SHA384
extraVolumes:
- name: admissioncontrolconfig
hostPath: /etc/kubernetes/admission-control-config.yaml
mountPath: /etc/kubernetes/admission-control-config.yaml
pathType: "File"
readOnly: true
- name: eventratelimit
hostPath: /etc/kubernetes/eventratelimit-server-namespace-user.yaml
mountPath: /etc/kubernetes/eventratelimit-server-namespace-user.yaml
readOnly: true
pathType: "File"
- name: audit
hostPath: /etc/kubernetes/audit-policy-global.yaml
mountPath: /etc/kubernetes/audit-policy-global.yaml
readOnly: true
pathType: "File"
- name: audit-log
hostPath: /var/log/kubernetes/audit/
mountPath: /var/log/kubernetes/audit/
readOnly: false
pathType: "DirectoryOrCreate"
apiVersion: kubeadm.k8s.io/v1beta3
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager:
extraArgs:
profiling: "false"
terminated-pod-gc-threshold: '10'
dns: {}
etcd:
local:
dataDir: /var/lib/etcd
imageRepository: k8s.gcr.io
kind: ClusterConfiguration
kubernetesVersion: v1.24.2
controlPlaneEndpoint: <control-plane host FQDN>
networking:
dnsDomain: cluster.local
serviceSubnet: 172.16.0.0/16
podSubnet: 192.168.0.0/16
scheduler: {}
---
apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
cgroupDriver: systemd
eventRecordQPS: 10
tlsCipherSuites:
- TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
- TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
- TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305
- TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
- TLS_RSA_WITH_AES_256_GCM_SHA384
- TLS_RSA_WITH_AES_128_GCM_SHA256
...
---
apiVersion: kubeadm.k8s.io/v1beta3
kind: JoinConfiguration
patches:
directory: /etc/kubernetes/patches
discovery:
nodeSelector:
matchLabels:
node-role.kubernetes.io/worker: ""
...