Additionally, every time you move your eyes, the high-detail area has to shift with them. If you did foveated rendering without low latency and high refresh rate, you're going to get visible pop-in of render detail levels.

Haven't tried it, but I'd hazard a guess that it would be really uncomfortable.