Note: I'm aware U.S. Govt 1st amendment doesn't restrict anything but the govt. I think we're both talking about the general concept of free speech, not that particular example.
Let's take the example of twitter. People swamp twitter with misinformation and toxic behavior. As individuals, we're all free to ignore/block the annoying/misleading people, and the company (the higher power) does not interfere. Ideal, right?
Except that the toxic people drive the others away. White supremacists attack and threaten people of color. Toxic bros attack and threaten women. Misinformation swamps the good information, and only the people that are already experts can tell the difference (and the misleading people will claim to be experts, so...).
If your response to above is "threats are illegal", you've not paid attention to how threats can made sufficiently arguable to be legal, but still clear enough to threaten.
Putting the burden on each individual to filter everyone is NOT taking "no" stance, it's taking the stance that those most willing to "speak" (which includes expressive actions) are given power. When the talking is for good things, that's great. But the people least willing to consider other people, the people lacking empathy, will have no restraint on what they say (or do, expressively) unless imposed. Long before they suffer any personal social repercussions they care about, they will have inflicted their harm upon a lot of people.
Preventing our "higher authorities" from taking action is reducing the power of collective people to take action. Sometimes collective people are wrong, but sometimes they are right, and that's why I say the topic isn't simple.