documentation states (Jobs | Kubernetes) for parallel jobs with a worker queue that:
- once any Pod has exited with success, no other Pod should still be doing any work for this task or writing any output. They should all be in the process of exiting.
Now I wonder why that would be? Just assuming that I have a job that should process 100’000 PDF documents in some way?
Assume I create a job with parallelism of 50. Further assume that all pods work on PDF documents with a single page, so that all goes fairly quickly; except the last document to be processed which is 1’000’000 pages.
Now there would be 49 pods alive and blocking resources until that last pod finished its work which might take ages…why would that be desired? Is there any Kubernetes internal reason that no pod should yet exit with success if other pods still do work?