You're leaving out some very well-established restrictions on free speech, including slander, libel, copyright infringement, obscenity, privacy violation... in short, absolutism hasn't been a prevailing philosophy for centuries. This sudden resurgence of it feels like a refusal to engage with the very real, very difficult debate on what speech deserves censorship.
It doesn't have anything at all to do with intelligence, but the observed consequences of certain kinds of speech. Lies, for example, that manipulate peoples' emotions. This is not unique to "stupid people."