That inherently arbitrary logic can be wielded against any regulation or ideology. Nearly every policy disadvantages some person or group's preferences/freedom to purportedly better enable a greater good.

This slippery slope reasoning would not only apply to your chosen target of "modern conservatism" but to anything under most ideologies and religions. As long as their implementation attempts probabilistically/historically led to a form of intolerance, suppression, and atrocity. Is Bernie Sanders also deserving of our intolerance because of the history of intolerance under socialism?