At one end of the continuum is an individual hanging up the phone when they decide they've heard enough from whoever called them. I think most of us would agree that this doesn't violate the principles of free speech, even if the caller was expressing valid ideas.
At the other end of the continuum is a government imprisoning someone for their speech. I think most of us would agree that this violates the principles (and laws) of free speech, except in certain circumstances (e.g. falsely shouting "fire" in a crowded theatre).
But there are cases in the middle of the continuum where things aren't so clear. If I run a small blog with occasional guest writers, a lot of us would probably agree that I can choose at will which content I publish without violating the principles of free speech.
But what if my blog has millions of viewers? What if it has become more of a "news site" or "forum" than a "blog?" What if it's the largest such site in the world, and the only one with significant market share? Perhaps then many of us would accuse me of violating the principles of free speech, since I have significant power to "silence" certain people or ideas.