It's not just "academics". It's anyone who can take an idea and successfully run with it, without stopping to notice all the signs that are yelling at them "Stop! This isn't working as you expected!" Academia merely happens to have a large concentration of such people, for structural reasons. (The "ivory tower" is a real effect, and it is both a good thing because it permits concentration and focus, and a bad thing, because it begets people too stuck in their theories and smart enough to "explain away" anything, until it is much too late.)

This is not something "the masses" do, because "the masses" do not have the ability to successfully run with an idea and gather enough power to impose it.

It's not just the holocaust, either. There's a huge history of 20th-century "social engineering" that rather a lot of people would rather forget, because it pretty uniformly went badly. See also "eugenics" for another reason that academics are afraid to think too hard about how people may be different or how genes may determine things about people... and not entirely without reason. "The masses" did not impose eugenics. In fact a lot of eugenics had to be hidden from the masses.

With power comes the power to screw up. Academics have a lot of power. It would be strange if they'd never screwed up.