Even then, if you stick to ALSA applications, you'll have a few issues:
1) You're going to be limiting yourself to a lot less possibilities (Skype uses Pulse, IIRC, for example)
2) You can't have simple settings such as per-application volume control.
Sure it "just works", but it does if you use a set of programs which only the people who say "it just works" (a lot of people, I know) restrict themselves to. While it may work for you, different people have different needs.
I'm still waiting for the day Linux audio (and video, but less hopeful because of proprietary drivers) simply works in the same way it does under Windows/Mac.
Footnote: I completely forgot to mention latency issues, which is awful for audio processing.