Gaming can't be good, because there's no controllers. Even if the hand and eye tracking is absolutely S-tier, you're never getting something like button input from swinging your empty hands around, twitching your fingers. Just the occlusion from the back of your hand would be enough enough for that to be impossible.
Fitness might be more doable, but even then I think you're going to be very limited with hand tracking only. I think there's a reason why Apple skipped over those segments almost entirely.
EDIT: I meant "VR" controllers, as in things that are tracked in 3d space. Of course you can use a game pad or MKB, but that kinda misses the point of being in VR in the first place.