Super interesting.

The TL;DR is that google built a system to shard client requests into different services/task by some largish shard key. The example they provide is to shard requests into ML model services by the underlying language (English vs Chinese) as different languages need different models.

Rather than requiring each product team to manage sharding, google built a custom layer to do it for any team to use. In practice it's not used by that many teams (~20 services at Google use it).