K8s Celery Deployment

Hey folks,

I’m wondering what is the best way to define deployment for things like Celery workers.

Essentially, Celery workers are basically long-running processes that start up and waiting for new tasks that they eligible to process.

While the obvious answer is to use the Deployment resource, it leaves an important question: how does k8s know that a celery worker is idle and it can be recreated with a new version? There is a readiness and health probes.

AFAIK K8s finds nodes that can be recreated by controlling traffic processing by each of the nodes, but what about cases like this when we talk about background processing

1 Like