This is a pretty common view for Vision Pro users. Hand tracking is great for these. Can't imagine having to use a controller.
Ofc Vision Pro users are extreme minorities so you are not wrong. But I highly encourage you to try out Vision Pro if you haven't.