The issues you are experiencing with the pointer are due to the fact that on X11, the pointer is on a separate thread. This is not the case on Wayland currently, but I remember seeing some work being done about it.
There is a super nice demo there that sheds light into what I'm trying to convey.