Academic freedom does matter, yes, but Academia isn't an ivory tower of neutrality. It shouldn't be. First because it should not be shielded from the consequences of its output, second because it should help advance mankind -- a political aim in more than a few regards.
Economics for instance has shown in multiple occasions that it is biased, toxic, and lacking diversity, pushing on towards ideology [1]. Academic Freedom does matter, but freedom appears vacuous when it only grants -- quite literally in terms of research -- the same ideas, the same people, the same social groups the whole spotlight.