I sympathize, brother.
My most irksome comments were "So... Giving mom the day off?" whenever someone would see me with my kids.
The most egregious thing to happen, was being kicked off of the local meetup for playdates... I assume I was approved to join the group because my first name is gender neutral.
Sometime between posting my profile pic (which included my kids) and attending my first meetup, I was removed from the group.