You can rent a 32 TPUv3 pod from Google Cloud at $32 per hour. So 128 pod would be roughly 150 per hour. $1K gives you 8 hours of training time.

https://cloud.google.com/tpu/pricing#pod-pricing