Pods can't communicate in Kubernetes

General Discussions
, ,
1

I have two pods. One pod (main) has an endpoint that queries other pod(other) to get the result. Both pods have services of type ClusterIP, and the main pod also has an ingress. The main pod is not able to connect to other pod to query at the given endpoint.

image
image1872×714 33.5 KB

In the above image, / endpoint works, but /other endpoint fails.

Below are the config files:

# main-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: main-service
  labels:
    name: main-service-label
spec:
  selector:
    app: main # label selector of pod, not the deployment
  type: ClusterIP
  ports:
  - port: 8001
    protocol: TCP
    targetPort: 8001


# other-service.yaml
apiVersion: v1
kind: Service
metadata:
  name: other-service
  labels:
    name: other-service-label
spec:
  selector:
    app: other # label selector of pod, not the deployment
  type: ClusterIP
  ports:
  - port: 8002
    protocol: TCP
    targetPort: 8002

All the docker images, deployment files, ingress etc are made available at: this repo.

Note:

  • I entered the other pod using kubectl exec, and I am able to make curl request to main pod, but not vice versa. Not sure what is going wrong.
  • All pods, services are in default namespace.

Relevant stackoverflow post: networking - Pods can't communicate in Kubernetes - Stack Overflow

2

Have you walked hrough this sort of debugging?

https://kubernetes.io/docs/tasks/debug/debug-application/debug-service/

3

yes. That doesn’t seem helpful in my case.

4

Can main pod ping other pod by the pod IP?

5

No. Main pod is not able to ping other pod (or service to be more specific) by ip. However the following are happening:

  1. Main pod is able to ping and get result on itself: curl http://localhost:8001/. curl http://main-service:8001 does not work from main pod.
  2. Other pod is able to ping and get result on itself: curl http://localhost:8002. curl http://other-service:8002 does not work from other pod.
  3. Other pod is able to ping and get result from main pod: curl http://main-service:8001 or curl http://10.105.46.163:8001
6

No. Main pod is not able to ping other pod (or service to be more specific) by ip.

You changed what I asked and then answered the revised question. Services do not forward ICMP (ping).

Can main pod ping other pod by the pod IP?

The purpose of the doc I linked you is to establish which layer is not working. The first step is to verify the fundamental networking (pod to pod, no services).

7

@HARSH_RAJ i am facing the same issue. which solution worked for u? could you please help it post here. Thankyou.

8

@HARSH_RAJ Please post solution here, if possible