It definitely can hurt. It comes up more than you might think. Some managers see it as a distraction, flight risk, potential time theft, etc.