I posted on reddit about how you could build this using open-source components: http://www.reddit.com/r/golang/comments/23zjxr/new_super_fas...

tl;dr use noVNC in the frontend and the websocket proxy bit is easy.