I once jokingly said it to an employee and immediately felt bad about it. Did a quick all-hands (we're a small company), apologized and said it wouldn't happen again.