kubernetes.
But I can't quite join the dots to see how you could have a distributed job processing system with just k8s.
Would you need to use some other system (perhaps also running in k8s) to track the queue of tasks and store task status & task results? Or can you get k8s itself to act as the task queue?
For us, we settled with Netflix Conductor as it scaled pretty well and allowed us to have pretty complex workflows and error paths and retry logic. Also is an independent and standalone tech.