Traffic is not going to hpa pod replicas

General Discussions
1

Hi Team,

We have setup ssh tunneling for our application through guacamole pod which opens a websocket to a guacd pod.
The setup information is as follows,
Cluster - EKS
guacamole - setup with Nodeport service, front ended with alb (through ALB ingress controller)
guacd - ClusterIP service type
guacamole will send the request to guacd service.

So In order to handle the traffic, we have configured hpa on our guacamole and guacd pods, both pods are having resource limits for cpu and memory. Here the problem is, when the guacd pod reaches threshold limit, new pod gets spinned up but the traffic is not going to the new pod, Instead the traffic goes to the first pod itself making it cpu exhausted and the connection get stuck/terminated from guacamole to guacd pod.
I am pasting the yaml files here. Request you to suggest your views on this issue.

apiVersion: autoscaling/v2beta2
kind: HorizontalPodAutoscaler
metadata:
name: pid-dm-rd-guacd
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: pid-dm-rd-guad
minReplicas: 1
maxReplicas: 10
metrics:

  • type: Resource
    resource:
    name: cpu
    target:
    type: Utilization
    averageUtilization: 80
  • type: Resource
    resource:
    name: memory
    target:
    type: AverageValue
    averageValue: 400Mi

apiVersion: apps/v1
kind: Deployment
metadata:
name: pid-dm-rd-guacd
labels:
app.kubernetes.io/name: pid-dm-rd-guacd
helm.sh/chart: pid-dm-rd-guacd-0.1.0
app.kubernetes.io/instance: pid-dm-rd-guacd
app.kubernetes.io/managed-by: Helm
spec:
replicas: 1
selector:
matchLabels:
app.kubernetes.io/instance: pid-dm-rd-guacd
template:
metadata:
labels:
app.kubernetes.io/instance: pid-dm-rd-guacd
spec:
containers:
- name: pid-dm-rd-guacd
image: “guacamole/guacd:1.3.0”
imagePullPolicy: IfNotPresent
ports:
- name: guacd
containerPort: 4822
protocol: TCP
resources:
limits:
cpu: 400m
memory: 500Mi
requests:
cpu: 200m
memory: 250Mi

The same yaml applicable for guacamole also.

Cluster information:

Kubernetes version: 1.21
Cloud being used: (put bare-metal if not on a public cloud) : AWS
Installation method: eksctl
Host OS: ubuntu
CNI and version: v1.9.0
CRI and version:

Thanks
Ramani

2

@Ramanichandran

Have you gotten any solution then please share with me also.
I am also facing same issues