Summers' claim is not incompatible with this observation. Not only disagreeing with Summers/Pinker but questioning their fundamental standing as "good faith actors" on these grounds is sad, but unfortunately pretty common. Your assertions about the grounding or lack thereof of these ideas in 2005 are simply false, and there's a reason why Summers is still remembered as an egregiously noteworthy case of incipient cancel culture.
This seems to be the standard middlebrow recourse for having to deal with uncomfortable ideas - find a shoddy, overconfident "debunking" of the inconvenient expert view from a trusted source (this will often rely on obvious misconstruals of the claims that the expert actually made), then call the experts "bad faith actors" when they continue to espouse said views.