Honestly I expect that as time goes and people wisen up, the more serious and legit a personality is the less likely they will be to accept an invite to that show until it is just bro-ey comedians and controversial personalities that need a steady flow of more supporters so they will say yes to the opportunity to expose their ideas unchallenged. Controversies sell well and Rogan is aware of that, at the end of it he only cares about growing his wealth and not freedom of speech.
BTW, I believe almost no one wants to change the amendments and remove freedom of speech. Those characters will still have the means to communicate their ideas and will be protected by the law, matter fact I myself have donated to ACLU who have defended the KKK in the court of law for the higher purpose of defending freedom of speech and rightly so. See https://www.aclu.org/press-releases/aclu-em-defends-kkks-rig...
I'm assuming fans of Rogan will be happy to read my comment since as Rogan promotes, we should give a platform to anyone or any idea and let people decide by themselves. Or that doesn't apply to his dissenters?