I'm not sure the point is valid. I certainly believe in freedom of speech and I accept my duty to protect the right of even the most offensive people to be heard (in fact, they should receive a greater degree of protection). However, by supporting a voting system that gives them no chance of being elected before they convince enough people for their ideas to be considered mainstream I'm not disenfranchising them since I'm not denying them the right to vote. I'm just marginalizing them, which is the proper response to all crazy claims and ideas. I offer two hypotheticals in support for those who still feel I'm infringing on someone's liberties by writing what I did:
(1) Suppose we live in a country that's a nuclear power. Suppose also there is a fascist party completely bent on nuclear war with Israel (for example) in our country and that they have a constituency that amounts to 1% of the electorate. Suppose nuclear war means total destruction on a global scale. I'll argue that giving those people a chance at forming a government is immoral and, in fact, disenfranchises everyone outside of your country since they have to control over who you elect, yet they suffer the horrible consequences.
(2) Suppose my party ran on a platform of dismantling democratic institutions (including elections of any sort) and vowed to distribute the majority of resources to my constituents, basically re-instituting feudalism. Random selection would give us the opportunity and the power to dismantle the democracy it was meant to be the engine of. It would also technically make it illegal for other countries to intervene, since our "feudal" government would be democratically elected.