Philips had a working concept with a beamer some years ago. It was called the "Drag Draw". Can't find much about it but here is a movie of the device in action: https://www.youtube.com/watch?v=7WsPncFFrRw
I think we learned a lot of HCI lessons from that project, mostly about guerilla arm. Smart boards can already do what you're looking for but they're insanely expensive and far from portable which is what we we're tasked to "beat".
Oh, forgot the best part, they were networked ,~20 tables at a time and used a server to facilitate connections and store sessions between each other.
Point being, what you're looking for exists but is expensive or requires some custom development of your own. I'd open source mine but my grant won't let me do that.