All controls for visionOS also have something called "direct touch" where if you reach out and press the button the button gets pressed!
Further, you can absolutely make 3D objects able to be grabbed, thrown, resized, etc. and it is actually pretty simple to do.