Changing the pointing device from default touch mode to PS/2 mouse in Machine>Settings>System fixed that for me. Seems to be the default for Ubuntu machines in recent versions.