It's not just the mistakes, when I speak with my non-standard accent or slightly incorrect phonetics the listener needs to work harder to process information, even if unconciously, and this on it's own already is detrimental or activates some bias.