Podcasts are normally plain mp3 (or similar) files that get downloaded as-is off an rss feed, as far as I understand. I don’t think anyone gets extra ads outside the sponsored/host-read ones.
If the same podcast is uploaded to Youtube through the uploader's official channel, it won't contain those ads and you're better off downloading that.