I think the part "Leveraging in-house primitives" is the important bit here. It sounds like they have an in-house task queue, so introducing another external one like Celery might add a lot of operational complexity rather than piggy backing on existing infrastructure. As for the reason for the existing infrastructure not being Celery, I suspect at Dropbox's scale tools like Celery start to break down.
> Or you could, like, install Jenkins, write your tests, and do all this without writing your own distributed job queue system.
They likely haven't written their own job queue, they've just used an existing one. Again, the operational overhead of Jenkins is unfortunately very non trivial.