Flight sims aren't terribly mainstream, sure, but I wouldn't be surprised if there's enough of a crowd between flight and racing sims who are willing to spend small fortunes on peripherals to keep VR going at some scale.
For gaming generally, there's a lot of friction that'll have to be removed before mainstream success is a possibility, I think--reducing the weight of headsets, reducing the cost, solving motion sickness, improving the game selection, improving situational awareness with the real world, reducing the setup effort, etc.
A lot of current games aren't worth the effort of dealing with all that, at least not once the novelty of VR wears off.