It matters a lot for libraries. It matters much less for applications.

I think it matters. Most people still follow common sense rules with it, e.g. bellow 0 is expected beta, minor version changes don't break stuff etc.