Is the nodePort number stable after it is created?

General Discussions
#1

My question refers to the situation when you ask the system to choose a NodePort number for you when creating a NodePort service. Once the service is created and remain undeleted (i.e. not re-created/updated), does the allocated NodePort number change during the service’s lifetime?

My understanding is, if there is no new node joins the pool, , there is no need for the NodePort number to change at all, since the NodePort number should have been allocated all all nodes in the pool, and the forwarding from that NodePort to pods have been configured for all current nodes in current pool.

However, if a new node joins to pool (for example, if the user decide to need a larger pool, or if a node is down and the system decides to bring in anther node so that the number of nodes in the pool remain unchanged), how about the NodePort numbers that are already allocated on existing nodes? Is it possible that some of the already allocated NodePort number on this new node is not available so that k8s system has to update the service associated to those NodePort number with new NodePort numbers, therefore causing NodePort number not stable during the lifetime of the service?

From a user’s point of view, he didn’t delete/update the service, so the service semantically has only been started once and has been running, this is different from pods which are expected to come and go.

Searched documents and can’t find an answer.

Thank you very much.

Wei

#2

Node Port never changes unless you deallocate it. The nodeport range is assumed to be owned by kubernetes on all nodes.

#3

Thank you very much for your reply. I can understand that when no new nodes join the pool, the Node Port will never change, because it has been allocated on all participating nodes already.

You comment: “nodeport range is assumed to be owned by kubernetes on all nodes.”, does that mean that when bring in a new node into the cluster, k8s makes sure that the all ports in the range is available when the new node joins? Therefore, when a new node joints the pool, the port number should also be available?

And the Node Port range is completely owned by k8s, i.e. it is not possible for any port number in that range be used by something non-k8s?

I really think that comment (i.e. “nodeport range is assumed to be owned by kubernetes on all nodes”) makes perfect sense, and that was an assumption I used when implementing a feature for work, however, I was challenged by our senior engineering director and he though that NodePort is unstable (my understand is it is dynamic but stable once created). However, I was not able to convince him and I was not able to find something authoritative, like documentations or something like that. Is there something like that I can reference? You know it is not that easy to convince someone higher up just to say “k8s should behaving like this and someone on the internet also told me so”. :slight_smile:

Thank you very much!

#4

| deeperblue
June 2 |

  • | - |

Thank you very much for your reply. I can understand that when no new nodes join the pool, the Node Port will never change, because it has been allocated on all participating nodes already.

You comment: “nodeport range is assumed to be owned by kubernetes on all nodes.”, does that mean that when bring in a new node into the cluster, k8s makes sure that the all ports in the range is available when the new node joins? Therefore, when a new node joints the pool, the port number should also be available?

It’s not checked, but it’s assumed. The node is joining the cluster, those NodePorts must be available. Given how NodePorts are implemented (iptables, usually) we will get triggered before any app that is listen()ing anyway.

And the Node Port range is completely owned by k8s, i.e. it is not possible for any port number in that range be used by something non-k8s?

It’s possible. Don’t do that.

I really think that comment (i.e. “nodeport range is assumed to be owned by kubernetes on all nodes”) makes perfect sense, and that was an assumption I used when implementing a feature for work, however, I was challenged by our senior engineering director and he though that NodePort is unstable (my understand is it is dynamic but stable once created). However, I was not able to convince him and I was not able to find something authoritative, like documentations or something like that. Is there something like that I can reference? You know it is not that easy to convince someone higher up just to say “k8s should behaving like this and someone on the internet also told me so”. :slight_smile:

In this case I am not just “someone on the internet” but the designer and author of the NodePort mechanism and a core kubernetes maintainer. I don’t like to play the “don’t you know who I am” card, but in this case, maybe it’s for the greater good :slight_smile:

If NodePorts were to change, it would be disastrous for Cloud LBs which are often implemented in terms of them.

I am on mobile this week, so I can’t easily trawl docs, but if it is not stated, it probably should be.

Tim

#5

Really appreciate your help. Two points from your comments are really very strong arguments to help people understand why node port should be stable:

  • The nodeport range is assumed to be owned by kubernetes on all nodes.
  • If NodePorts were to change, it would be disastrous for Cloud LBs which are often implemented in terms of them.

I have an additional related question.
Is NodePort service creation synchronous in terms of the nodeport number allocation? I am talking about programmatically creating the service using go client library. After creating the service, I immediately query the service to find out the allocated node port number, should it always be immediately available? (Well I guess the update to iptables on all participating nodes may not be synchronous).

Thank you very much!

#6

The NodePort is allocated synchronously today, though I don’t think I
would advise to depend on that. We’re looking to add new APIs that
serve the same purpose but are factored better (called Gateway) and
there’s some value to allowing async allocation (more like
load-balancer IPs). Service will almost certainly not BREAK the
current assumption, but I don’t want to guarantee that Gateway will
retain it.