It is always interesting to get different perspectives from folks.

How do you filter the people who hold the door for everyone vs the ones that are only doing it for ladies? Or for those fellas who where told countless times as a youth to hold the door for a woman. Because someone is polite, that is a red flag?

As for calling a woman a lady this one is new to me, and I don't think I've ever used the word "colleague" aloud in my life. I would say, "that is a {guy|lady} that I work with." I've never been made aware previously that referring to a woman as a lady would be taken negatively. I had only heard that some woman don't care for ma'am as they can feel that is a title for an older woman. fwiw, I do tend to use gender neutral words like "they" or "them" over he/she when writing.