I have to wonder how many who claim to immediately tell the difference between mp3 and FLAC have instead simply been exposed to horribly encoded mp3s.
Although this is anecdotal, it pings off your own experiences: A friend of mine (and self-described audiophile) recently got into a discussion with a mutual friend regarding mp3, FLAC, and how both of them could tell the format apart simply by listening. I suggested that perhaps the mp3s he had available were simply poorly encoded, and sent him a sample of my library that's a direct rip from a CD (at a modest bitrate of either 192kbps or 256kbps, I can't remember). He then realized that the mp3s he had been comparing against his FLAC copies of the same album were likely YouTube rips from before he bought the album, and that he likely never downloaded the official mp3s when he purchased the FLAC sources. He confided that he had a difficult time determining which was which when comparing the two on more equal footing with the sample I sent.
I still posit that the average person can't tell the difference, even with above average hardware. Exceptionally sensitive hearing outside the normal range of most humans notwithstanding, a decently encoded mp3 (or Opus format for that matter) should hardly be distinguishable from a FLAC, if at all. I suspect that most of those who claim they can hear the difference aren't particularly special--they're just comparing poorly encoded files against a source format.
At any rate, I'm in my mid-thirties, and I know my hearing isn't quite as acute as it was a decade ago. FLACs are more of a novelty for the sake of original sources, because there's no way I could tell the difference between that and mp3s with a good bitrate. With the exception of the Hunt for Red October soundtrack, I've not been able to tell low/moderate (~128kbps) bitrate mp3s apart from high (384kbps) ones AFAIK (the high pitches in the hymn "warble" at lower bitrates--not sure I can detect this now, though).