Using incorrect language, however, makes whatever you are writing about seem less thought-out or even just plain wrong.
Have you ever read 4chan? You can argue that that's English, since it's English speakers using the English language... but you will surely agree that their version of English makes them look like uneducated morons, even without considering what they are trying to say.
Anyway, the style guide here suggests that authors use the clearest word to describe a given situation or quality. I don't see anything wrong with this -- clarity is always good. (Your writing could still be understandable without using the words correctly, but why use incorrect words just because some other people are also wrong?)