Is there any method implement scalable job?

General Discussions
1

I’m a workflow data processing development. In our user cases, user spend long time and large resource processing data. So the ablility to auto scale user processor workload is necessary.

Now we are using Deployment to deploy, and autoscale by some metric info. But when scale down workload may cause kill some running Pods, causing waste of resources.

Job may a better choose for our cases, but Job cannot autoscale. Is there any other choice?

2

It’s not an immediate answer, but Elastic Index Jobs are in flight (beta release targeted for 1.27 in April) that might work for your use case:

3

Thanks, I will study what and how about these first.