Carmack famously quipped that "I can send an IP packet to Europe faster than I can send a pixel to the screen" [1]. For VR and AR to break through, they're going to have to be in a form factor that resembles normal glasses, and at a much higher render quality (4K+) than anything we've got now. That's almost certainly going to require rendering remotely Stadia / Xbox Cloud style, with some minimal local processing.
[1] https://superuser.com/questions/419070/transatlantic-ping-fa....