Because it's a colloquialism, and because despite that "No is a complete sentence" post from earlier this week (last week?), it often comes across across as rude, unnecessarily harsh, and over time will make people think of you as an asshole.