I don't understand this obsession with big sizes watches maybe it's because I actually don't wear a watch but I've ordered the 38mm model (and I'm a man). My reasonning is that the last thing I want is something big on my wrist.
I mean, it's a totally valid perspective, but holy edge-case
If I have a watch as a fashion accessory/jewelry, being small is no longer a design parameter, and in fact being larger might be seen as positive- like a large diamond ring.
Watches are not always worn just to tell time.
That's not to say Apple can't reverse that trend - they hold considerable sway in matters of fashion and design, but right now this is the dominant form for men's watches.
See also the complaints about it not looking expensive enough. It doesn't matter if it works, if its durable, or even if it actually is expensive, the only selection criteria was appearing expensive to others. That way, people you don't know or care about will be impressed by you. That guy I don't know who I passed in the hallway would be extremely intimidated if my watch band looked handmade, and thats very important to a certain caliber of people.