DNS Resolution not working for busybox

development
k8s-blog
coredns

#1

I am testing my DNS Service and following the guide from link : Debugging DNS Resolution - Kubernetes

As per the steps, I have created a pod busybox and then executed the command which tests the DNS service. But I am getting the following error.

kubi@kubem:~/wordpress2$ kubectl exec -ti busybox -- nslookup kubernetes.default
Server:    10.96.0.10
Address 1: 10.96.0.10

nslookup: can't resolve 'kubernetes.default'
command terminated with exit code 1


kubi@kubem:~/wordpress2$ kubectl exec busybox cat /etc/resolv.conf
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local lan
options ndots:5

Please help me to investigate this further. My kube-system pods are in running state.

Here is the PODs I am running on the system with all other parameters :slight_smile:

kubi@kubem:~/wordpress2$ kubectl get all -o wide -n kube-system
NAME                                       READY   STATUS    RESTARTS   AGE   IP               NODE     NOMINATED NODE   READINESS GATES
pod/coredns-86c58d9df4-dxr5l               1/1     Running   92         12d   10.244.0.37      kubem    <none>           <none>
pod/coredns-86c58d9df4-xc4gg               1/1     Running   94         12d   10.244.0.39      kubem    <none>           <none>
pod/etcd-kubem                             1/1     Running   12         12d   192.168.56.110   kubem    <none>           <none>
pod/kube-apiserver-kubem                   1/1     Running   99         12d   192.168.56.110   kubem    <none>           <none>
pod/kube-controller-manager-kubem          1/1     Running   122        12d   192.168.56.110   kubem    <none>           <none>
pod/kube-flannel-ds-amd64-bpkhg            1/1     Running   7          12d   192.168.56.112   kubew2   <none>           <none>
pod/kube-flannel-ds-amd64-j2ww9            1/1     Running   8          12d   192.168.56.111   kubew1   <none>           <none>
pod/kube-flannel-ds-amd64-v5hb2            1/1     Running   15         12d   192.168.56.110   kubem    <none>           <none>
pod/kube-proxy-f697j                       1/1     Running   7          12d   192.168.56.111   kubew1   <none>           <none>
pod/kube-proxy-sbsfv                       1/1     Running   12         12d   192.168.56.110   kubem    <none>           <none>
pod/kube-proxy-v5k4p                       1/1     Running   7          12d   192.168.56.112   kubew2   <none>           <none>
pod/kube-scheduler-kubem                   1/1     Running   143        12d   192.168.56.110   kubem    <none>           <none>
pod/kubernetes-dashboard-57df4db6b-xgnsx   1/1     Running   110        12d   10.244.0.38      kubem    <none>           <none>
pod/tiller-deploy-dbb85cb99-7nr2l          1/1     Running   7          12d   10.244.2.51      kubew2   <none>           <none>

NAME                                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)               AGE     SELECTOR
service/kube-dns                                           ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP         12d     k8s-app=kube-dns
service/kubernetes-dashboard                               ClusterIP   10.104.199.119   <none>        443/TCP               12d     k8s-app=kubernetes-dashboard
service/prom-prometheus-operator-kube-controller-manager   ClusterIP   None             <none>        10252/TCP             7d23h   k8s-app=kube-controller-manager
service/prom-prometheus-operator-kube-dns                  ClusterIP   None             <none>        10054/TCP,10055/TCP   7d23h   k8s-app=kube-dns
service/prom-prometheus-operator-kube-etcd                 ClusterIP   None             <none>        4001/TCP              7d23h   k8s-app=etcd-server
service/prom-prometheus-operator-kube-scheduler            ClusterIP   None             <none>        10251/TCP             7d23h   k8s-app=kube-scheduler
service/prom-prometheus-operator-kubelet                   ClusterIP   None             <none>        10250/TCP             7d23h   <none>
service/tiller-deploy                                      ClusterIP   10.107.118.136   <none>        44134/TCP             12d     app=helm,name=tiller

NAME                                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                     AGE   CONTAINERS     IMAGES                                   SELECTOR
daemonset.apps/kube-flannel-ds-amd64     3         3         3       3            3           beta.kubernetes.io/arch=amd64     12d   kube-flannel   quay.io/coreos/flannel:v0.11.0-amd64     app=flannel,tier=node
daemonset.apps/kube-flannel-ds-arm       0         0         0       0            0           beta.kubernetes.io/arch=arm       12d   kube-flannel   quay.io/coreos/flannel:v0.11.0-arm       app=flannel,tier=node
daemonset.apps/kube-flannel-ds-arm64     0         0         0       0            0           beta.kubernetes.io/arch=arm64     12d   kube-flannel   quay.io/coreos/flannel:v0.11.0-arm64     app=flannel,tier=node
daemonset.apps/kube-flannel-ds-ppc64le   0         0         0       0            0           beta.kubernetes.io/arch=ppc64le   12d   kube-flannel   quay.io/coreos/flannel:v0.11.0-ppc64le   app=flannel,tier=node
daemonset.apps/kube-flannel-ds-s390x     0         0         0       0            0           beta.kubernetes.io/arch=s390x     12d   kube-flannel   quay.io/coreos/flannel:v0.11.0-s390x     app=flannel,tier=node
daemonset.apps/kube-proxy                3         3         3       3            3           <none>                            12d   kube-proxy     k8s.gcr.io/kube-proxy:v1.13.3            k8s-app=kube-proxy

NAME                                   READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS             IMAGES                                          SELECTOR
deployment.apps/coredns                2/2     2            2           12d   coredns                k8s.gcr.io/coredns:1.2.6                        k8s-app=kube-dns
deployment.apps/kubernetes-dashboard   1/1     1            1           12d   kubernetes-dashboard   k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1   k8s-app=kubernetes-dashboard
deployment.apps/tiller-deploy          1/1     1            1           12d   tiller                 gcr.io/kubernetes-helm/tiller:v2.12.3           app=helm,name=tiller

NAME                                             DESIRED   CURRENT   READY   AGE   CONTAINERS             IMAGES                                          SELECTOR
replicaset.apps/coredns-86c58d9df4               2         2         2       12d   coredns                k8s.gcr.io/coredns:1.2.6                        k8s-app=kube-dns,pod-template-hash=86c58d9df4
replicaset.apps/kubernetes-dashboard-57df4db6b   1         1         1       12d   kubernetes-dashboard   k8s.gcr.io/kubernetes-dashboard-amd64:v1.10.1   k8s-app=kubernetes-dashboard,pod-template-hash=57df4db6b
replicaset.apps/tiller-deploy-69ffbf64bc         0         0         0       12d   tiller                 gcr.io/kubernetes-helm/tiller:v2.12.3           app=helm,name=tiller,pod-template-hash=69ffbf64bc
replicaset.apps/tiller-deploy-dbb85cb99          1         1         1       12d   tiller                 gcr.io/kubernetes-helm/tiller:v2.12.3           app=helm,name=tiller,pod-template-hash=dbb85cb99
kubi@kubem:~/wordpress2$

Here are the logs for both coredns pods :

kubi@kubem:~/wordpress2$ kubectl logs -f coredns-86c58d9df4-dxr5l -n kube-system
.:53
2019-03-12T08:37:33.029Z [INFO] CoreDNS-1.2.6
2019-03-12T08:37:33.029Z [INFO] linux/amd64, go1.11.2, 756749c
CoreDNS-1.2.6
linux/amd64, go1.11.2, 756749c
 [INFO] plugin/reload: Running configuration MD5 = f65c4821c8a9b7b5eb30fa4fbc167769
 
kubi@kubem:~/wordpress2$ kubectl logs -f coredns-86c58d9df4-xc4gg -n kube-system
.:53
2019-03-12T08:37:35.359Z [INFO] CoreDNS-1.2.6
2019-03-12T08:37:35.359Z [INFO] linux/amd64, go1.11.2, 756749c
CoreDNS-1.2.6
linux/amd64, go1.11.2, 756749c
 [INFO] plugin/reload: Running configuration MD5 = f65c4821c8a9b7b5eb30fa4fbc167769

Kubectl Version:

kubi@kubem:~/wordpress2$ kubectl version
Client Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:08:12Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.3", GitCommit:"721bfa751924da8d1680787490c54b9179b1fed0", GitTreeState:"clean", BuildDate:"2019-02-01T20:00:57Z", GoVersion:"go1.11.5", Compiler:"gc", Platform:"linux/amd64"}

Mysql and Wordpress Kubernetes installtion giving Name Resolution Error