You may not have been paying attention, but unfortunately that's not the case.
https://www.thefire.org/list-of-campus-disinvitations-2000-2...
http://www.spiked-online.com/newsite/article/whats-new-about...
http://www.mediaite.com/online/university-president-shuts-do...
And I can link to many more examples if you wish. It's moved from "your opinion is incorrect but I respect your right to speak it" to "I don't agree with what you say so I'm denying your right to express it".
Now do you understand the problem? It's quite incredible, actually.