Imagine a male, non-native English speaker learning from a group of stay-at-home moms. Deep South, Midwest, take your pick. He's gonna pick up some stuff. Men in the US aren't expected to say "Aw honey," or "Bless yer heart!"
It would entirely depend on how you said it. Like sure, if you said it a flamboyantly feminine way that would be strange to some people, but you could easily say those things in a masculine way.
I don't think there is any situation where a ESL speaker is going to be told that they are speaking like the wrong gender.
There are plenty of others, but yes, not as many as some other languages.
That's the entire point. That you could say those things in a masculine way.
Anyone can train themselves to speak in ways that make them sound either closer to the gender the present as or the gender they'd like to present as.
Consider the tone, cadence, and word choice of a "hard cowboy" compared to a "dweeb streamer". They sound different, no?
Consider the vocal fry of the valley girl or the bouyancy of the "gay voice".
Hearing the following spoken over the phone, can you really not see that one might sound more like a women, and one more like a man?
"Hiya! Oh.Em.Gee! Okay! So! You're never ever going to guess what IIIiiiIII Saw! Today!" in a dynamic high pitch?
VS
"yo. wazzup. bro, I saw some shit today"
>I don't think there is any situation where a ESL speaker is going to be told that they are speaking like the wrong gender.
You might need to get out more. Or thank your fate that you're blessed to have never experienced it yourself, but even then I'd be surprised you've never had a friend experience it.
LGBT people grow up being told they're talking wrong all of the time. Women who prefer declaritve sentences are told they are being to manly all of the time.