I have a Macbook Pro M4 with macOS Sequoia.
I installed and configured 6 node cluster with 3 control and 3 compute using VMs on Virtualbox 7.1.8 on Centos9 ARM x64 Linux.
Cluster information:
Control:
control-01.kube.lab
control-02.kube.lab
control-03.kube.lab
Compute:
compute-01.kube.lab
compute-02.kube.lab
compute-03.kube.lab
I made the host entries and respective IPs on all hosts for internal DNS resolution.
Kubernetes version: 1.32
Cloud being used: bare-metal
Installation method: kubeadm init
Host OS: CentOS9 ARMx64
CNI and version: 0.3.1 from Calico v3.26.0
CRI and version: containerd containerd.io 1.7.27
I was able to perform kubeadm int and was able to join rest of the control plane and compute nodes successfully into the cluster. Installed Calico for CNI and Containerd for CRI. All services look good and kubectl get nodes shows all hosts.
KubeDNS is healthy with IPs and CoreDNS has right IPs and all pods running.
NAME READY STATUS RESTARTS AGE
coredns-668d6bf9bc-8rl2h 1/1 Running 0 112m
coredns-668d6bf9bc-mgzlv 1/1 Running 0 112m
kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 112m
NAME ENDPOINTS AGE
kube-dns 10.244.131.193:53,10.244.131.194:53,10.244.131.193:53 + 3 more... 112m
Although everything looks healthy, when I run a test-dns using busybox pod image, DNS resolution inside the pod fails. I can’t ping dns server and name resolution is failing.
I’m using Bridged Network adapter setup for VMs and I tried all modes of Promiscuous. Right now left it on ‘Allow All’. IPv6 is disabled.
Additional config steps I followed:
sudo swapoff -a
sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter
cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 0
EOF
sudo sysctl --system
Logs for both CoreDNS and Kube-Proxy don’t report anything and SeLinux is disabled. Just hoping anyone could point out what am I missing here. VM to VM pings work with even FQDNs. I can even reach internet from these VMs.
Please help!