I am a tall woman. If I wear "unisex clothes" -- and not wearing a bra seems a large factor here, so you cannot see bra straps (through clothing) from the back -- and have my hair short enough, I get lots of "Excuse me sir, I think you are in the wrong bathroom." This in spite of most of my life being read as extremely feminine.
Humans haven't solved this yet for humans. I imagine we are rather far off for software.
Recognizing any feature at all in any image is a matter of levels of success. There is no algorithm which will successfully find ever a person, let alone a specific person or specific type of person in an image 100% of the time.
There are algorithms within the opencv project which do a very good job of finding certain male or female characteristics in an image. If you combine them I would say you could get a reasonable accuracy of 70-80%. If this is good enough for your application, then you win!
https://flickrcode.files.wordpress.com/2014/10/xkcd-1425.png...
Perhaps not, but it sure would be funny to see the results. Imagine a vending machine or other public machine that spoke to you in different languages (or dialects) based solely on how you look. That would give a good laugh. At the same time, someone would get offended.
I'm pretty sure we have technology at this particular level, even if it's not readily available to the public. Distinguishing sexes, on the other hand, is a vastly more difficult task than recognising a handful of trained stimuli. Some humans have trouble reliably distinguishing sexes, especially when it involves people from ethnicities one has little or no experience with. Many toddlers, in fact, would have trouble telling sexes apart from mere photographs -- except where obvious clues like beards and stereotypical clothing are present.
Edit: Then again, it really depends on the level of accuracy and compatibility you are looking for. Training a system to look out for stereotypical features (hair style, lipstick, earrings, beards, clothing colours, height, body contours) is definitely possible with current technology.