It turns out this is kind of already in the Linux Kernel. See here: https://lwn.net/Articles/614348/
You can tunnel TCP / IP / GRE, reasonably even Ethernet itself over UDP. This is actually really interesting for cases where you want layer 2 bridging over the internet without all the complexity and overhead of a VPN.