That’s not because he necessarily said anything actually sexist or actually racist, but because he said the kinds of things that you usually only hear from sexist people, and in particular, the kind of sexist people who are also racist.
That is the big thing I saw as well.
I think there is an analogy to the body's defenses against viruses. The body recognizes certain shapes from the outside surface of viruses and attacks anything sharing those shapes. The memo was chock full of ideas and phrases that match those used by the worst kind of bigots.
If you look at the plain meaning of the words it isn't so bad. But most people's response is based on the pattern-matched shape evaluation instead.