https://papercutsoftware.github.io/teleportme/ was an open-source-ish solution I came across recently (built on FaceTime), but I haven't tried it. Seems like they're making similar design decisions as you.