Cluster information:
Kubernetes version: 1.13.11-gke.14
Cloud being used: GKE
Installation method: GKE
Host OS: Container_optimized
I am trying to build the exact infra mentioned here https://medium.com/google-cloud/kubernetes-hpa-autoscaling-with-kafka-metrics-88a671497f07
Basically, the article is specifically for scaling pods based on the reading the Stackdriver metrics(stackdrive receives traffic from Kafka server and scaling happens based on it.)
Everything is working fine until I create a producer deployment and it works fine as expected. It is able to connect to Kafka server but when I run the consumer deployment, it is failing with following error in the container logs.
% Error: 10.128.0.9:9092/bootstrap: Connect to ipv4#10.128.0.9:9092 failed: Operation timed out (after 128991ms in state CONNECT)
screenshot https://prnt.sc/q7hwvs
I just have one GKE cluster with 2 nodes + one Kafka compute engine instance. There is no special setup. I am just following the above article.
I don’t know what should I debug because producer is working but the consumer is not.
If anyone can help that would be really appreciated.