Something that I wish I had known about Apache Kafka a year or so ago is that it essentially has no support for long-running tasks, i.e. tasks where longest-possible-worker-execution-time >> longest-tolerable-group-rebalance-time.
After much angst in trying to work around this issue, I finally gave up and switched to Pulsar. Pulsar isn’t without it’s own issues (mostly around bugs and general maturity) but it handles this particular scenario admirably.