I agree with everything you've said except for "and that would break the magic". We've already seen proofs of concept of "how natural does this kind of thing seem", in Star Trek and so on, and I can easily imagine not even noticing this kind of user error after a couple of minutes. Even if the screen burst out in simulated blue sparks when the person on the other end reached past the camera, I think that… probably doesn't break the immersion?

You "just" need conversation to be truly real-time, seriously low latency, and with really good audio and visual quality. Once you've got those, I can easily imagine crossing the threshold where you're speaking through a magic pane of glass rather than into a screen. Think of how you might find interacting with a literal magical artefact that did this.

You can see a bit of the visual degradation and artifacts when reaching toward the 'screen' in the video at 1:46/1:50, as the woman with a baby on her lap reaches her palm toward the screen.

I can't quite tell if they're doing this in the video, but you can project things out of a 3d display. You just have to avoid the object touching the sides of the screen. With that, you could have participants feel like their hands are in the same place without hitting glass.