No, it didn't work for anything higher than 720p. For those Youtube requires using DASH + MSE.
What am I doing "wrong"? What am I missing here?
Did you enable media.mediasource.enabled in about:config? Check it for Firefox 36, and if it's true, see if it's highlighted or not. If yes (and you are using the stock Mozilla build and not Gentoo custom one) it means you don't have the default.
Sure, you could enable it already for a while there, but it was buggy. For me the default was false. Strangely, it's false even in Firefox 37, which contradicts the release notes. But at least it works well now in actual videos when enabled.
As you can see, media.mediasource.enabled is disabled by default.