This is where GCP's GCS (Google Cloud Storage) shines.

You don't need to mess with prefixing all your files. They auto level the cluster for you [1].

[1] https://cloud.google.com/storage/docs/request-rate#redistrib...