a lot of difference which CAN be noticed results from bad encoders and the generic settings used - not strictly related to compression but to other aspects of the psychoacoustic model. classical music encodes better with one set of params, heavy metal with another. there were times when 160kbps has been transparent for me from the original using hi-fi, DACs and headphones. on recording, 24 bit makes a difference vs 16 bit, on output not so much in today's nil-dynamic range records.
telling the difference often requires constantly comparing to the original master where certain aspects sound just slightly different (not necessarily worse), and do not justify a 3x increase in compressed size to get perfect.
the biggest disappointment with a lot of lossy music is bad encoders, encoder settings, poorly (re)mastered originals and non-existent dynamic range at the source rather than a limited quality distribution format.