What I've learned from 27 years of working in subfields where Bernstein is prominent is that there are elements of his reputation that are probably well-earned. I used to believe (with, I think, good reason) that people who had viscerally negative opinions of him were betraying something important (and bad) about themselves. That might have been true at the time (call it '96-'06). It's less true now.
Regardless: Bernstein is important because of Curve25519, Ed25519 and ChaPoly. And those are huge contributions, and more or less not up for debate, unlike his C code.