Well, no, we say server, because server is gender-neutral. Compare steward/stewardess with flight attendant. And some servers will get mighty angry if you call them a waitress.
I don't necessarily think that response is justified when no offense is actually meant. But the point is that there has been an identifiable trend in our language towards moving away from gender-specific terms for roles that shouldn't otherwise have a relevance to gender.
I'd argue this movement is overdue for epithets. I think "asshole," "asshat," "assclown," really anything in the venerable ass family, makes for an ideal gender-neutral epithet. We should use these more often!
Edit: typo