It all depends on how sensitive you are to these things, I've done such blind tests, and to my ears the fall in quality was significant, even painful.
I can also hear differences between lossless/lossy encodings. If I tweak encoding parameters until I don't notice a difference, there is usually no real space saving afterwards - so why not go with lossess instead?
My comparisons were made several years ago using LAME 320 CBR, LAME VBR, OGG, lossless (WAV) 16bit 44.1kHz and lossless (WAV) 16bit 48kHz. I believe at least most professional musicians and sound engineers would be able to identify the difference between all of these; while they might not always be "worse", their sonic character is certainly different.
I.e. I will definitely fail ABX test, but I still feel difference. IMHO, ABX tests are wrong tests for that problem.