There's a player called HereSphere that works a whole lot better than any other VR video player on the market. Nothing I have tried comes even close to the level of smoothness, even over SMB on a crappy external drive.

https://heresphere.itch.io/heresphere-vr-video-player-quest-...

The demo is limited only by a nag screen if you want to check it out.