Dominant Resource Fairness (DRF) in Kubernetes?

development
#1

Hi all

I would like to know, Is it possible to implement and develope Dominant Resource Fairness (DRF) in Kubernetes? it has already been implemented in Apache mesos but I’m not make sure that it is possible in Kubernetes.

Cheers

0 Likes

#2

Is that not captured within the Resource Policies? Managing Compute Resources for Containers - Kubernetes

0 Likes

#3

In resource management, it says that when we create a pod, we can specify resource requests. However, in DRF since the objective is to maximize the allocation of each user however, in kubernetes it seems fairly vague. Also, I don’t know how to implement DRF allocation policy in the Kubernetes since it needs calculations.

0 Likes

#4

The closest things I have seen is by adding resource quota’s to a namespace Configure Memory and CPU Quotas for a Namespace - Kubernetes .

You can then limit user access to a particular namespace to give it a similar effect. There may be another way to achieve what you are looking to do but I am not familiar with it.

0 Likes

#5

Kubernetes itself does not align that well with DRF. It really is intended to be explicit about resource allocation.

That said, you could work on something to dynamically adjust resource quotas for namespaces to give them a percentage of a cluster as @macintoshprime suggested. Scaling workloads in those namespaces would likely require some additional plumbing.

At Pod level you might be able to achieve something similar by feeding a custom metric to the horizontal pod autoscaler or vertical pod autoscaler.

0 Likes

#6

I don’t know why I didn’t think of this when typing up that reply (came to me right after), but take a look at kube-batch. They have DRF implemented specifically for jobs using it as a scheduler.

1 Like

#7

Thank you so much for your response. It seems complex but I will investigate it to find out a way to implement it.

1 Like