I haven't been bothered over them yet, but I fully expect a conversation where someone yells at me for "filming", I show there's no cameras, and they insist they're simply hidden.
It's mostly curiosity, people wondering what they are. They expect them to be camera glasses but believe me when I tell them they aren't.
Never heard this one before. And my family cares about being rude.