I'm seriously considering trying something like this just to see if it works. TCP would kill it though (maybe?). But maybe UDP plus some clever ffmpeg invocation...
https://bbs.archlinux.org/viewtopic.php?id=43896
(circa 2008 people were doing this... semi-unironically ;-)