Scheduling-according-to-the-available-memory-of-the-node

General Discussions
1

Asking for help? Comment out what you need so we can get more information to help you!

Cluster information:

Kubernetes version: :v1.14.4
Cloud being used: (put bare-metal if not on a public cloud)
Installation method:
Host OS: centos7
CNI and version:
CRI and version:

How to set it up. When the available memory of the node is less than 8G, the node is set to be unschedulable, and when the memory is restored to more than 8G, it will automatically become schedulable.

When the node is set to be unschedulable, the above pod continues to run unaffected

Or set the pod so that the pod is not scheduled to a node with less than 8G memory,Because the memory required by different pods is very different, it cannot be solved by setting the request

I set the configuration file of node. evictionHard:memory.available: 8Gi. but The pod running on the node will be evicted when the available memory of the node is less than 8G

The essential requirement is to leave 8G margin for each node for the pods running on this node. Because the memory demand of the pod may suddenly increase during the running process, 8g is reserved as a buffer

2

Have you looked into resource requests and limits yet?

4

Thanks for your answer, but specifying requests and limit is not suitable for my case. Different pods have different memory usage and it is not known when the pod is created. Some only need 100M but some need 6G. It would be a waste of resources if all pods set 8G requests

5

Sounds like the way you want to implement your solution does not match the pattern Kubernetes was designed for. The tools for scheduling are taints, tolerances, affinity, priority classes, resource limits, and resource requests. This lends to maintaining groups of nodes and needing to set expectations for the workloads being deployed (setting resource requests and limits). In the case of the latter, you just throw your workloads at a cluster and trust the scheduler to place it somewhere appropriate based on your description.

A controller can be made to do this, or do whatever you want in your cluster. Though I might recommend reconsidering your thought process on why you think the solution you need this for makes sense compared to the way Kubernetes was designed.

If the constraint here is just that you don’t know the workload sizes being deployed, perhaps using something like virtual kubeletes to just run the workload on single use cloud servers. This is something I favor for adhoc jobs where the resource usage is really not foreseeable because the workloads are controlled by end users. It’s also possible to better secure this type of workload.

6

kubectl get nodes -o custom-columns=Name:.metadata.name,AVAILABLE_MEMORY:.status.allocatable.memory,AVAILABLE_CPU:.status.allocatable.cpu

1 Like
7

You can use this to get nodes available memory and available cpu