So, some things happened in the twentieth century. Basically there are two trends that developed, and reinforced each other without necessarily intending to.
One was the disengagement of general audiences from "classical" (for lack of a better term) music. The other was the disengagement of composers from general audiences.
Each group did this, largely for its own reasons, but each side's disengagement only sped up the other's. The result of this is that you have a general audience who is largely ignorant of what was going on in music, and a world of composers who became more and more insular, writing largely for each other and to make increasingly-arcane points about music theory.
This means that when you listen to contemporary classical music, you're A) listening to something that was not written with you in mind as an audience, and B) utterly lacking the background -- the traditions, the movements, the reactions and counter-reactions -- to understand what's going on in the music.
In a broader sense, this happened in basically all the arts, but music is one of the areas where we seem to notice it more often.