What happens when the user moves the cursor outside the window, and then clicks? Or moves the cursor into a hot corner of the screen, activating something like a screen saver?

Deltas are necessary but are only part of the solution.