Latency is also critical for VR and AR
It would certainly be a much more complex solution that would still have inferior results compared to computing everything locally (just like cloud streaming now). Just that it's not "change the laws of physics" impossible to get to a workable implementation.
See also John Carmack's comments from 2013 on latency mitigation: https://web.archive.org/web/20140719053303/http://www.altdev...