It's this and one other point: Games that people aren't bored of in an hour.
To me, very few games have come out for VR that don't feel like gimmicky experiences. Even Half Life Alyx, as advanced as it was, kinda felt like a theme park ride after a while. I'm not sure if there's technical reasons for it, but it feels like nobody is taking VR development seriously.
It's hard to justify strapping a TV to my face and feeling uncomfortable for one-off experiences. Even if there was a game with some depth and replayability, I would be even more annoyed to play it on such an uncomfortable headset.
Almost everyone I know is not using their VR headset anymore. I'm not sure it will ever move past that phase, because people want it to be smaller and, simultaneously, more technically immersive. So we're in some weird in between zone where it's neither.