Will glad if you can help me here with this.
Kubernetes version: 1.24.9
Cloud being used: bare-matel
Installation method: kubeadm
Host OS: Ubuntu-22.04
CNI and version: Calico-typha 3.25.1
CRI and version: containerd
I recently added more nodes to the cluster, which has more then 50 worker nodes, and 3 VMs which are control-plane nodes.
This cluster is being upgraded from k8s version 1.17, up to now with 1.24.
Calico-typha is using default encapsulation setup, with IPIP for all traffic.
After adding the new set of servers to the cluster, DNS stopped working, but only for PODs scheduled on the new nodes.
Tests show can all tcp/udp traffic from new nodes to PODs on the existing nodes, fails.
ping/ICMP works with no issue.
I do see that the packets arrives to the destination PODs, but blocked by IPtables, under calico chain, as “ctstate INVALID”
What could be the issue?
I thought migrating from IPIP calico setup to vxlan encapsulation, to mitigate any other BGP related issues that might come from other surrounding network setup which uses BGP.
Thanks in advance!