The impression I get is that when Windows executes the "Found new hardware" thing, it installs (usually from Windows Update, if the setting is enabled) the drivers, but it's as if the drivers were installed for that USB port only... it appears to install the driver again and again as I move the device to other ports. This is not the way Linux deals with device drivers (once a driver exists in the kernel or is available as a kernel module, it works irregardless of the port), and as far as I know on OS X and BSD kernels it's similar.