Of course it's a fact about the document. Damore does not say this. If you claim he did, you should easily be able to prove it by quoting him saying it. No one has done that, because the accusation is false. The burden of proof is on the accuser. The accuser is not presumed to be telling the truth on the basis of their social standing, gender or the emotional intensity of their reaction.
>But here's the deal, a bunch of other really smart people think it did do exactly what you claim it didn't. Now what? Are they wrong, you're right? On what basis?
On the basis of the content of the memo, they are wrong.