Even I couldn't reliably determine which word was replaced given screen coordinates, and I spend half my time pushing pixels around (web design & photography).
Yeah, in order for this to be usable, it would need a better UI. Maybe some kind of overlay: it could automatically position the annotations at the correct position relative to the video, and make them appear and disappear at the appropriate times.
I wonder if implementing something like that would be technically feasible.
You are proposing rebuilding the existing feature Google wants to turn off.