But as it is, you have the freedom to speak, to collaborate with others to speak, and to gather more resources to speak more effectively--and the government may not restrict you from doing so.
And what is the alternative? Allow the government to decide how much speech you may have? How much speech is worth? Catalog and index everyone's speech to ensure they aren't exceeding their quota? Disallow citizens from freely collaborating and associating for the purpose of speech? That's not freedom. We know where censorship leads.
One can recognize the drawbacks of big PACs and corporate political spending, while at the same time recognizing that those are natural consequences of liberty.
As they say, freedom of speech is the worst--except for the alternatives.