Dilemma PV máx per node in Kubernetes
Hi all,
I’m a relative newbie to using Kubernetes for production workloads.
We want to use a cluster to migrate a number of applications that are currently running on another server using Docker.
Analysing our needs, most of these applications do not consume a large amount of RAM/CPU and use volumes (more than 30 different ones).
This allows us to run many of these small services on the same node with Docker. Since we are at the limit of our possibilities in terms of HW, that’s why we are considering moving to Kubernetes. Our customer works with Azure cloud so we have chosen Azure AKS.
Here starts our headache, Kubernetes/ AKS (as well as others like AWS EKS) has some restrictions on the maximum number of PVs (data disks) that can be connected to a node. Here is the general specification (Node-specific Volume Limits | Kubernetes) (In our particular case we have a maximum of 16 PV).
Things I wonder:
- If our pods need PV, having 30 different applications, how can I only map 16 (even 4-8 on other lower node types)?
- Is there a different strategy other than sharing PV, through PVC and subPath, for the different applications? I need to understand how you all have designed that data access layer with Kubernetes. NOTE: I’m sure you’re wondering why we can’t transform these StafulSets to (Deployement)Stateless. We don’t have an external database. It’s not that kind of application.
- Taking into account the previous point, let’s imagine that I group applications by some criteria… If I use the same PV through PVCs for the different applications, how does Kubernetes handle when it has to schedule the movement of these pods to another node? Either because of some error or because of Pod anti/affinity + taints.
Please, I understand that this issue is probably something basic, but we are just starting to use Kubernetes in production and frankly it’s something beyond my logic.
I look forward to your comments 
Have a great day!