Y'all literally fought a war over that very question back in the 1700s.
Hell, is there any question less political than to what extent individual freedoms should be trumped by the state and the collective "good"? That, in its various forms, is the fundamental political question from which all other political questions follow.
I don't see another choice for them. The populist authoritarian streak taking hold in many country's right wing parties goes against much of what the ACLU stands for.
Part of the reason their name has weight, is their history of defending anyone including in at least one case actual neo-nazis.
IMO populism is anti human rights. I think that's the reason ACLU is no longer neutral. They were totally fine with traditional conservative principles for many decades.