First, you have to accept that the debate isn't about all about scientific facts; it's about their implication in workplace policies. Reducing workplace interaction to science is a fallacy.
When deciding how to act in the world, you have to use your judgement. When planning your diet, do you read a bunch of nutrition papers and decide what to eat 100% on what they say, or did you apply some cultural knowledge?
Second, "facts that are completely true" does not gibe with most cognitive scientists seem to agree.
Which is it? Did you check every fact yourself, or did you just accept what he had to say uncritically?
From the article: Similarly, if you can read this guy's entire memo and find yourself blindly nodding, then you probably aren't using your brain either.
The phrase facts that are completely true sounds young and naive. Just like the left can cite studies about unconscious bias, the right can cite studies about biological differences. That doesn't mean they're all true.
Even if you accept that "scientific facts" are the way to decide the workplace policy, his article isn't complete. Where is the scientific fact that says how much discrimination against women there is? He said "I'm not denying discrimination exists" but didn't address it further.
BTW I largely defended Damore, because many people I talked with have extreme "left wing" views. His points were completely mischaracterized in the media. It became a political circus. But I will also argue against extreme "right wing" views. That was the point of the quote.
FWIW I worked at Google for 11 years, and both my parents and sister have Ph.D.'s and are in the sciences, two in biological sciences. I know how "studies" work. Both people on the left and right (at least in this debate) have a naive view of science.