There are practical limits, sure. Most people are not going to want to turn their entire body around 180-degrees just to look at another window. Within those limits though I find that more space is generally better.
If one has to Alt+Tab or Command+Tab through programs, yes, that would suck the life out of productivity.
The only real benefit larger screen spaces (larger than a ~30 inch monitor) provide me with, are when I need about 4-6 open windows (maximized vertically) to switch between when dealing with large merges/patches. I haven't experienced any practical benefits beyond two ~30inch monitors.
I also maintain that this is significantly more intuitive than having to memorize keyboard shortcuts for each of my more frequently used programs.
I suppose everyone's workflow is different though. If you're comfortable with just one monitor then good for you. I for one will be a lot happier once I can arrange my workspace into a neat grid of windows arrayed across my desk.
I use a side by side in landscape orientation, so a 7680x2160 desktop. I think this is about ideal width and dot-pitch for me. Higher resolution in the same field of view would be wasteful as the pixels are already too small to see individually unless they are high contrast. I still prefer mild RGB sub-pixel anti-aliasing on fonts in this setting. A wider field of view would also be useless for desktop tasks since I would not be able to comfortably scan it all. While I wouldn't mind a taller field of view, I don't have the budget nor tolerance to stack monitors vertically nor to switch to an array of portrait orientation monitors.
VR will not give me any screen "real estate" unless it has incredibly high resolution and wide field of view or has solved eye-tracking and zero-latency foveal rendering. I am not interested in turning my head constantly to shift a high resolution porthole around a blurry overview or (worse) a blinkered black void.
Because of the gap between monitors, I sit slightly offset and favor the right half of the left monitor for transient tasks. I keep chat and email windows open on the far left and tend to open a browser or editor on the left half (or two thirds) of the right monitor, while opening reference materials like PDFs in the rightmost extreme.
I do have traditional virtual desktops where a hot-key press can shift my displays to show any one quadrant of a logical 15360x4320 desktop. This is a holdover from my days on a single, lower resolution screen. I find that I use it far less frequently now, as I can indeed use the large visible area to organize things related windows spatially. I use it mostly for mode-switching between different sets of windows which may keep the same arrangement for weeks at a time. For example, one quadrant is my code-editing and communication space and another is a deployment/test suite/log-following space. I have room to open other reference materials or sample data around my code editing, and conversely room for other SSH shells and state-spelunking tools around my test and log windows.