Cluster information:
Kubernetes version: 1.21
Cloud being used: Tested on AWS and GCP
Installation method: kops
Host OS: Ubuntu 20.04 (also tested on CentOS)
CNI and version: -
CRI and version: CRI-O
Hi,
We are developing an application deployed on Kubernetes, for which we use hugetlbfs as shared memory across pods within a node.
This was working fine but for some time now, most modern distributions have now support of hugetlb cgroup controller and Kubernetes has HugePages management, so unless we miss something:
- a pod must have HugePages resources limit/request set, otherwise it cannot create hugetlb allocations at all
- this resources cannot overcommit, so this limit/request must be split across all pods.
We use the hugetlbfs backed with files, that any pod can create, so there is no “by-pod” management here, and we don’t figure out how to configure this properly.
For now, we disabled the hugetlb cgroup controller on the host. It looks like a bad idea to us to have to customise the host.
Is there any good pattern to handle that on Kubernetes?
Thanks in advance for any advice on this.