# Rescheduling pod after scale up **URL:** \ **Category:** General Discussions\ **Created:** [February 2, 2022, 11:13pm UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967 "2022-02-02T23:13:23Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![mario\_martinez](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/mario_martinez/32/9078_2.png) [@mario\_martinez](https://discuss.kubernetes.io/u/mario_martinez)\ **Post date:** [February 2, 2022, 11:13pm UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967/1 "2022-02-02T23:13:23Z")
I have a question, I hope you can help me We have multiple deployments of the same JAVA based application with two pods distributed in our two nodes AKS, both at 90% of RAM (Request limit), to improve performance, I add a third node. No pod of the deployments is rescheduled to the new node, they stays in the old two nodes and when we have load peaks, the service is affected, since during a very short period, there are peaks of 95% of RAM in the nodes and we suspect that is affecting us What should I do, so that without redeploying the deployment, kubernetes reschedule some of the pods to the new node that has no load? We have this RAM limit in the pods resources: limits: memory: 1024Mi requests: memory: 1024Mi Thank you very much for the answers ### Cluster information: Kubernetes version: 1.21.2 Cloud being used: Azure Host OS: Linux ---
**Author:** ![ga2022](https://avatars.discourse-cdn.com/v4/letter/g/6f9a4e/32.png) [@ga2022](https://discuss.kubernetes.io/u/ga2022)\ **Post date:** [February 3, 2022, 2:43am UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967/2 "2022-02-03T02:43:22Z")
You could use Node selector/affinity to schedule the pods on the new node. In addition, you can drain the nodes with high RAM. > **[Assigning Pods to Nodes](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)** > > You can constrain a Pod so that it can only run on particular set of Node(s). There are several ways to do this and the recommended approaches all use label selectors to facilitate the selection. Generally such constraints are unnecessary, as the... ---
**Author:** ![mario\_martinez](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/mario_martinez/32/9078_2.png) [@mario\_martinez](https://discuss.kubernetes.io/u/mario_martinez)\ **Post date:** [February 3, 2022, 8:33am UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967/3 "2022-02-03T08:33:01Z")
Hi ga, thanks for the answer I know about the affinity and selector, but i want something automatic. I mean, is not posible to kube scheduler, reschedule pod if the nodes are stressed? I was looking for information but i didnt find anything ---
**Author:** ![Theog75](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/theog75/32/9241_2.png) [@Theog75](https://discuss.kubernetes.io/u/Theog75)\ **Post date:** [February 3, 2022, 12:18pm UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967/4 "2022-02-03T12:18:30Z")
the kubelet (on the node) will evict pods if: 1. the node is scarce in memory and needs to free up memory 2. a kubectl drain was issues. as @ga2202 mentioned - you can use pod antiAffinity to make sure two pods will not deploy on the same node - but the only way to move a pod from one node to another is to kill the old pod and redeploy a new pod instead (on a new node if configured). so when you mention rescheduling, you can do that (any change to the deployment yaml will cause a redeploy of the pods) but it will kill the pods and redeploy them. I do not know the exact architecture you are implementing but memory consumption does not indicate stress (as opposed to cpu which can throttle) the worst that can happen memory wise is that your pod will be oom killed if it exceeds its memory limit. ---
**Author:** ![mario\_martinez](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/mario_martinez/32/9078_2.png) [@mario\_martinez](https://discuss.kubernetes.io/u/mario_martinez)\ **Post date:** [February 3, 2022, 1:22pm UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967/5 "2022-02-03T13:22:49Z")
Hi theog, first of all, thanks you for the answer In short, it is not possible for kubernetes to move the load of pods already deployed, as long as the node is working properly Even if one node has 90% ram and another 30% I supose at the end, i have to redeploy the app so the resources will be balanced ---
**Author:** ![Theog75](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/theog75/32/9241_2.png) [@Theog75](https://discuss.kubernetes.io/u/Theog75)\ **Post date:** [February 3, 2022, 1:47pm UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967/6 "2022-02-03T13:47:08Z")
redeploy will not redistribute the pods necessarily, if the node has enough memory un-requested - the pod might deploy on the same node again, you can drain the node and that will evict the pod (move it to a different node). I am trying to understand what does stress as you mentioned above means ---
**Author:** ![xavi](https://avatars.discourse-cdn.com/v4/letter/x/8e8cbc/32.png) [@xavi](https://discuss.kubernetes.io/u/xavi)\ **Post date:** [February 6, 2022, 11:07am UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967/7 "2022-02-06T11:07:39Z")
The [Kubernetes scheduler](https://kubernetes.io/docs/reference/command-line-tools-reference/kube-scheduler/) only checks the node’s load (among other things) before it schedules the pod on the more suitable node. Once the pod is scheduled in one node, for the duration of its lifetime, it is bounded to that node (see [Pod Lifecycle](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/)): > Pods are only [scheduled](https://kubernetes.io/docs/concepts/scheduling-eviction/) once in their lifetime. Once a Pod is scheduled (assigned) to a Node, the Pod runs on that Node until it stops or is [terminated](https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#pod-termination). So the answer to your question is “no”, as others had already mentioned: the pod will not be re-scheduled to any other node. Maybe you would like to consider using an _horizontal pod autoscaler_ (see [Horizontal Pod Autoscaling](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/)) to increase the number of pods in a deployment if the CPU or memory usage goes over a threshold. So, if your application is heavyly used, new pods of your application will be created to share the load (assuming you have available resources on the existing nodes or you manually add new nodes to your cluster for new pods to be created). So, in your scenario, when your application is stressed, the HPA will try to create a new pod; the scheduler will check if any of the nodes have at least your pod’s requested memory available (1024Mi). If a third node is available, assuming that the current nodes has a RAM utilization of 95% and they will not have 1024Mib avaliable to host the new pod, the scheduler will deploy a new pod on the third (empty) node. This will increase the number of pods from 2 to 3, resulting in one pod of your application on each node (it will not move one pod from from, say node 2 to node 3). This will distribute your application’s load between 3 pods, instead of 2 and will decrease the load on your application’s pods. If the load then goes down, the HPA can also reduce the number of pods in your deployment. Using the _cluster autoscaler_ (see [Automatically scale a cluster to meet application demands on Azure Kubernetes Service (AKS)](https://docs.microsoft.com/en-us/azure/aks/cluster-autoscaler)), you can increase the number of nodes in your cluster automatically when they are stressed. So, using a combination of an HPA for your application and the cluster autoscaler, you can have a fully “elastic” solution for both your app and your cluster. ---
**Author:** ![mario\_martinez](https://sea2.discourse-cdn.com/flex016/user_avatar/discuss.kubernetes.io/mario_martinez/32/9078_2.png) [@mario\_martinez](https://discuss.kubernetes.io/u/mario_martinez)\ **Post date:** [February 7, 2022, 1:42pm UTC](https://discuss.kubernetes.io/t/rescheduling-pod-after-scale-up/18967/8 "2022-02-07T13:42:21Z")
Thank you all for the answers, we will work on the horizontal scaling to unload the saturated nodes and we will redeploy so that the scheduler reevaluates the nodes.