I don't think than X11/Wayland should work over network at all. They don't have enough information do this efficiently. On other side it works REALLY well on toolkit(GTK/Qt) level. Try GTK Broadway
http://wiki.c2.com/?FrescoFramework http://berlin.sourceforge.net/
Hard to believe that 15 years has already passed since I heard about it.
On the one hand, VNC suddenly just became obsolete. On the other hand... Ubuntu Core is playing with getting Webkit-GTK running directly on top of DRI (via Wayland), and I hope like mad there's never an alternate reality where Chromium takes the place of X11... eep
(I mean it already kind of has, with Electron :S)