DNS resolution fails within the cluster, and it can only resolve the Pods deployed on the same host

Kubernetes version: 1.20.0
Installation method: kubeadm
Host OS: BigCloud Enterprise Linux release 8.2.2107 (Core) (Like Centos)
CNI and version: flannel Quay
CRI and version: docker 19.03.5

Sorry, friends. I have encountered a problem regarding DNS.

I have set up a cluster and deployed our production application. However, there is an issue with connectivity between microservices. Upon investigation, I found that our services can not access the Eureka registry service using the internal cluster domain name.

The Eureka service is deployed on the nb-17-8 server. Pods deployed on the same server, nb-17-8, can access Eureka using DNS. For example, the three replicas of Eureka need to communicate with each other for synchronization (which works fine):

ndsp          eureka-0                                       1/1     Running                 1          71m     10.244.86.42    nb-17-8     <none>           <none>
ndsp          eureka-1                                       1/1     Running                 1          71m     10.244.86.44    nb-17-8     <none>           <none>
ndsp          eureka-2                                       1/1     Running                 1          71m     10.244.86.45    nb-17-8     <none>           <none>

Within the Eureka pods on nb-17-8, they cannot access 10.96.0.1 or 10.96.0.10:

kubectl exec -it -n ndsp eureka-0 -- sh
/ # nc -v 10.96.0.1 443
10.96.0.1 (10.96.0.1:443) open
^Cpunt!

/ # nc -v  10.96.0.10 53
10.96.0.10 (10.96.0.10:53) open
^Cpunt!

On other hosts, the Service IPs are accessible:

[root@nb-17-57 ~]#  nc -v 10.96.0.1 443
Ncat: Version 7.70 ( https://nmap.org/ncat )
Ncat: Connected to 10.96.0.1:443.
^C
[root@nb-17-57 ~]#  nc -v 10.96.0.10 53
Ncat: Version 7.70 ( https://nmap.org/ncat )
Ncat: Connected to 10.96.0.10:53.

On the hosts, accessing the Eureka pod via domain name is not possible, but using the Pod IP works (from all hosts and pods):

[root@nb-17-57 ~]# curl -I 10.244.86.42:8761
HTTP/1.1 200 
Content-Type: text/html;charset=UTF-8
Content-Language: en-US
Transfer-Encoding: chunked
Date: Wed, 29 May 2024 11:14:14 GMT

In the CoreDNS pod, there are the following error messages:

[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:44838->8.8.8.8:53: i/o timeout
[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:51686->8.8.4.4:53: i/o timeout
[ERROR] plugin/errors: 2 eureka-2.eureka. A: read udp 10.244.0.11:54068->8.8.4.4:53: i/o timeout

All hosts have the following resolv.conf file content (since it’s a closed internal network, domain name resolution uses the hosts file and there is no internal DNS server):

cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script

Does anyone have any good solutions?

I use AI tools for translation, please forgive my poor English.