Slow CoreDNS queries response and timeouts

General Discussions
1

Cluster information:

Kubernetes version:

kubectl version
Client Version: v1.29.14
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.29.14

Cloud being used: bare-metal
Installation method:
Host OS: AlmaLinux 8
CNI and version: Flannel ver: 0.26.4
CRI and version: cri-dockerd ver: 0.3.16

Hi, for Google APIs I’m getting slow CoreDNS responses and timeouts, I have tried for days to discover where the error might be without luck, any suggestions to fix this issue?

I am using the node:18-alpine image which has alpine ver 3.21.3

Thanks.

1 Like
2

When I run this commands it works but then it stops working when I reboot the server:

nft flush ruleset

nft add rule ip nat POSTROUTING ip saddr 10.244.0.0/16 ip daddr != 224.0.0.0/4 masquerade

nft add rule ip filter FORWARD ip saddr 10.244.0.0/16 ip daddr 10.96.0.0/12 accep
3

If run this command:

[root@master k8s]# kubectl exec -it -n server-api server-55d7fdc87b-hpfwm -- nslookup google.com 10.96.0.10
nslookup: write to '10.96.0.10': Host is unreachable
;; connection timed out; no servers could be reached

command terminated with exit code 1

I get this reply

[root@master ~]# tcpdump -i any host 10.96.0.10 -n
dropped privs to tcpdump
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on any, link-type LINUX_SLL (Linux cooked v1), capture size 262144 bytes
22:38:23.848680 IP 10.244.0.102.46942 > 10.96.0.10.domain: 40402+ A? google.com. (28)
22:38:23.848680 IP 10.244.0.102.46942 > 10.96.0.10.domain: 40402+ A? google.com. (28)
22:38:23.848838 IP 10.96.0.10 > 10.244.0.102: ICMP host 10.96.0.10 unreachable - admin prohibited filter, length 64
22:38:23.848846 IP 10.96.0.10 > 10.244.0.102: ICMP host 10.96.0.10 unreachable - admin prohibited filter, length 64
4

Hi,
Are you trying to connect to CoreDNS pods from the master node?

5

Hi! I’m trying to get my NodeJS API app to resolve the Google APIs OAuth2 address. If I replace the CoreDNS Service IP address (10.96.0.10) with the CoreDNS Pod IP address it resolves but it does not work with the Service IP address.

Also If I run these commands, in the master node, it starts working with the Service IP address, the one saved in the Pod /etc/resolv.conf:

nft flush ruleset

nft add rule ip nat POSTROUTING ip saddr 10.244.0.0/16 ip daddr != 224.0.0.0/4 masquerade

nft add rule ip filter FORWARD ip saddr 10.244.0.0/16 ip daddr 10.96.0.0/12 accept
6

I added this to the pod’s deployment config yaml file and it started working:

  spec:
    dnsPolicy: "None"
      dnsConfig:
        nameservers:
          - 8.8.8.8
          - 8.8.4.4
          - 1.1.1.1

I understand that I am not fixing the routing problem but at least my API NodeJS app can resolve the google apis addresses.