This is great, I was working on something similar in the last few days, but since it is hard to cover every podcast, I stopped to think of a way to niche down. I feel your pain with GPU and scalability to transcript podcasts.

I was thinking of adding something like this for the UI https://github.com/johan-akerman/SpotifyTranscripts in case you find it useful.

Good luck! It is a really nice project.