I think generally it's easy to know you won't be needed by other patrons in a coffee shop, or that their chatter won't require you to weigh in or judge, so your brain just keeps an ear out for normal danger noises.

At work, headphones off, my brain is subconsciously trying to keep me informed about the noise around me in the workplace, so that I can respond if needed (generally I'm not) not helpful, brain. :P