So long as there are women-only gyms, scholarships, etc this argument falls a little flat. In other words, you can't have your cake and eat it.
That is, once you've eat the cake, you can no longer hold it in your hands. It's (generally) used to refer to a situation where one stance or course of action (naturally) precludes another.
So I think I agree with you, but the actual wording is "who gets to make the decision matters" rather than "reasons matter".
That said, there are milder scenarios for discrimination and there are severe ones. This one is severe in its stupidity.
But honestly, it seems like you're just nitpicking here because you don't want chicks at your LAN party. Some of us find that pretty hateful.
If a "women's" gym is the closest to his house and has the equipment he uses (e.g. I didn't know treadmills were gender-specific) then why shouldn't a man go? His money's the same as anyone else's.