Also built something slightly similar - without the need to interact https://github.com/frans-fuerst/track Works on Linux with XOrg and Windows only unfortunately..