I’ll start by saying that I’m sorry if I’m posting in the wrong category.
Secondly, I’ve just started looking into Kubernetes and I find it quite interesting but I’ve hit a bump. I’ve tried to run a cluster on 3 nodes (1 master and 2 workers) and I’ve just launched some nginx pods. I am using MetalLB and have created a service to be able to access the pods. The issue I’m seeing is that I’m limited to a single host’s bandwidth (1 Gbps in this case). One of the worker nodes are saturated while the other has no traffic on port 80. How can I scale the network throughput? Is this possible (I suppose it is but I haven’t found any articles / posts to help with that).
I had 4 machines running a wget to the metallb’s IP address, so multiple streams. The file they were all pulling was a 1GB file. I can see the throughput going up to 1Gbps on one of the machines (node1) and about 700 on the other, but running a tcpdump on both nodes, only one of them is actually showing traffic on port 80.
From what I gather, since there’s a single IP for that service, it’s quite normal that the IP address is “binded” to one of the physical hosts but then again, I’m limited to that host’s bandwidth.
This is the case in layer2 mode and is unavoidable. You have more flexibility if you can use BGP/ECMP, but that requires your network to be setup to support it.
The alternative if you have to use layer2 mode is generally to designate a set of nodes as the “edge” nodes and their purpose is to act as a LB. These would have beefier nics likely not be tasked with running any other workloads.
The alternative if you have to use layer2 mode is generally to designate a set of nodes as the “edge” nodes and their purpose is to act as a LB. These would have beefier nics likely not be tasked with running any other workloads.
Ok, how do I set nodes as “edge” nodes? I mean, let’s say that I want to set 3 nodes as edge nodes so they would receive all the traffic. I would need to have a service on each one of them or …? Sorry if it’s a silly question, I have only started with Kubernetes but I’m trying to figure out if I can use it with some of my clients who would really benefit of the scalability of this system.
Traffic would not be spread equally amongst the edge nodes. It’s closer to a fail-over pair, traffic for a particular LB service would goto a single node. If that node should fail, it would failover to the 2nd node designated an edge node. They’re there to serve as the front end LB to the backend worker nodes which would have the work spread among them.
Ok so then I’d still be limited to the bandwidth of a single server. Got it.
Can you provide some info as to how should these nodes be set? Because, from what I know so far, you cannot “force” a service to be binded to a certain host, can you?
I’m back with a question, even though it might sound stupid. If I have an external load balancer (HAProxy) in front of my cluster nodes, I should be able to just use a nodeport to get the traffic inside the cluster, with HAProxy doing roundrobin, for example, right?
