We're pliable, we'll adapt. VR doesn't physically alter your body, so with repeat usage, we'll adapt. The garbage in processed foods is likely damaging us far far more, as those lab created ingredients actually become a part of us.
The case that used to scare the hell out of the lab techs I ran into was that of someone using a simulator at work for a few hours a day, but driving to and from the office. The context switching is far from instant, and acclimatising to it could be the eye based equivalent of giving yourself a calluses through repeat gentle stabbing with a needle.
Additionally, if you spend a good proportion of your waking hours in a device setup for you to focus at infinity, what happens when you take it off and need to focus elsewhere? The muscles won't have been exercised and you won't be used to using them when looking at objects you judge to be at different distances.
This is on top of the classic intro, which would be "welcome to VR - we're going to show you 20 new ways to make someone fall over with surprisingly little effort", and many of those don't even need head mounted displays.
As far as I could tell AR got a lot of early support largely because using see through displays eliminates most of the problems while preserving the most interesting non-gaming applications (i.e. keyhole surgery overlays on the patient) though the calibration requirements are insane.
Not to say you can't harm yourself with conventional setups, you can (epilepsy being the extreme example, but eye damage from using displays irresponsibly is very common in the geek world), just VR adds whole new dimensions to it that should really be far more openly discussed.