On what setup? All YouTube videos load and start playing instantly for me. Every time I've experienced otherwise in the last couple years, it's been my first indication that e.g. AWS is exploding that day
I suspect that non-Chrome browsers are being intentionally hobbled
I don't doubt at all that Google hobbles their sites on Firefox but at least on my machine they aren't doing a great job of it
If I set my user agent to something like Linux/Ubuntu, it loads just fine. If I set my user agent to some unheard Linux distro, it lags as the same with FreeBSD.