Binary compatibility is certainly a major issue but I think you are downplaying the compatibility headaches introduced by the way each distributions are solving the same issues with slightly different methods. I'm thinking of: logging, default character sets, locale, timezones, mount points & encrypted block storage, basic OS versioning information, networking, service management, networking. And I'm forgetting a plethora of others areas.
There were/are attempts to have standardization around some this, like the FHS or the LSB for example. But it's hard to argue that systemd was far more successful into bringing a semblance of order than any past attempts. There's still a long way to go.
The general lack of compatibility between distributions usually leads software publishers to packaging and testing only one or two versions of their software. The community will then try to repackage it for other distributions.
When the apps are not 100% open-source, it usually leads to huge delays for updates and broken packages after a while.