I know because before I was an investor, I was a founder. We'd never have wanted to work out of an investor's space. It would have been distracting and demoralizing.
I think framing matters. If it's presented as "when you take my money, you must live on my land, be overseen by me" etc., it's almost feudal, and of course no-one with free choice would choose that kind of environment. But if it's "if you want to be mentored by me, I can host you for a short while along with a bunch of other ambitious folks", it seems different.
Sounds like the thing to do would be to eliminate that friction by helping people quickly and easily locate places of their own, then.