The Linux kernel itself is a remarkably stable system as far as ABI/API is concerned (Linus you rock here), but the packages gyrating around it are completely impossibly unstable and unreliable.
A fine example of this is trying to get an SSTP client working with NetworkManager recently. The whole NetworkManager VPN thing is an epic pile of hacks with bits of UI and bits of backend sticking out all over the place. If you're using Debian stable for example, there is no chance in hell you can get a working configuration unless you go and fix the numerous broken interfaces all over the place manually and arguing with pkg-config and literally hundreds of dependencies. How the fuck this ever even worked to start with I don't know. There are massive API changes with minor versions of packages all through NetworkManager and Gnome, which is just wrong.
The underlying issue is simply shitty engineering from a large body of open source projects who don't care about upstream or doing their job properly. To name and shame: The entire Gnome project, just about anything Ubuntu have chucked out, the entire KDE project post KDE2.
It's utterly frustrating. I wish they did take a more BSD approach, which is conservative engineering i.e. think about it, get it right and leave it.
For all the people here whinging about innovation, waah waah, ship old versions etc, I doubt you've actually had to deal with these issues or build a project on top of them. There seems to be a large detachment from reality here.
As an example to back my point up, one of the reasons Windows is so damn popular is that the API is incredibly stable and software written in the mid-90s still works fine on current versions without a recompilation or any API changes. A two week old build of NetworkManager probably will break all the VPN providers...