Isn't this the crux of infamous "ad hominem" argument: attacking the person instead of their reasoning?
Even smart people sometimes make stupid arguments.
And good arguments can be made by anyone.
It's very difficult to be right 100% of the time.
It's also quite unusual to be wrong 100% of the time.
Evaluate the reasoning, not the author.
Stupid argument, not stupid person.
Look at what Alsup said to Boies.
Still, this is easier said than done.