Most podcasts though I listen to at infinite speed, meaning I don't listen to them at all. You can go through them 2-3x speed because there's really nothing there.
It's like code: if it's boilerplate you can just skim it but if it's really doing something you have to read it slowly multiple times. And just like the saying: nothing of value is easily gained.
If you find yourself speeding though things - in general, not just podcasts - I would ask myself if they're worth doing at all.