I wouldn't leave the mouse to run about, causing trouble in other's yards. Just kill it swiftly.