What's wrong with distraction? Plenty of people stay in shape as a side-effect of their enjoyment of sports and games. Am I not really working out when I play soccer or go rock climbing?
For example, I used to go running and weightlifting while listening to music and podcasts for example, because I was in the mindset of physical exercise == unpleasant & time consuming therefore I must distract myself "to get it over with". VR workouts sound a lot like an idea coming from that same mindset.