While its true that headphones are associated with hearing loss, there are lots of headphones out there that have noise-canceling capabilities without being excessively loud.
[1] By cancel out, I mean make it impossible to understand the conversation, while the headphones are silent or playing in low volume.
(NB. This reply entirely useless if you can't tolerate in-ear buds.)
http://productivity.stackexchange.com/questions/7213/do-nois...