Yeah, I haven't found a way around that. Hoping for "advanced window control" in a future version of visionOS. That and curved windows.
Although, it isn't a huge deal for me now. Walking around a physical space is part of the "new way of working" I'm exploring. Monitors beat the AVP if I'm going to be parked in a desk anyway, but the AVP unlocks a workflow in which I'm fully digital but also less tethered to a desk, which is interesting to me.
If Apple could find a reasonable solution for typing on a non-physical keyboard I'd be very happy. Not sure I want to resort to Keyboard Pants.