In an ideal system, all voters would digest accurate information and make informed decisions, but we know this is not the case. Media has bias and political ads are misleading... plus people don't have the time or impetus to spend vast amounts of time 'sorting the wheat from the chaff.' Many people aren't even skeptics... which is a requisite for being able to filter the signal through that noise.
Yes, there is a strong free speech argument for donating money to campaigns, but make no mistake... oligarchs would (maybe already do?) run our country without any restrictions in place. Go read about Thomas Jefferson and the history of the estate tax before you try to argue this ideology because it is simply wrong and a danger to democracy.