It is a bit condescending in my view to withhold that information from the clerk. But then again, I'm only making assumptions. Also, I've done similar things where I refused to let on my internal reasoning for various reasons. Sometimes just to mess with people. And clearly, many people thought me a fool for it. If I'd actually told them my reasoning? In many instances that would not have improved their opinion of me. So no loss.
There are also many instances where I could have benefited immensely from sharing my reasoning, because people could have corrected my mistaken assumptions. Can't say I've got it figured out when to keep shut and when to share.