"Breaking" is undefinable. I remember reading about some user complaining that a small bugfix broke their work setup because it fixed a bug that used to make the CPU go 100% when the spacebar was held. The user would hold it with a weight and get the CPU to make heat that way. The small fix broke his experience.

There are many other examples of this. Breaking compatibility by fixing bugs people rely on for instance.

Any change can break something for a user downstream. It's a very subjective evaluation for the producer to imagine potential impact to their consumers. It's easier if the relationship is rich and exclusive. In open-source where people do it on their free time and have thousands of consumers with widely different interests, it's pretty much impossible to label the release in a way that's conveying the right message to every user.

Semver, the way i see it, is just a way for the producer to subjectively label the amplitude of changes. The consumer should then ideally be familiar with the producer and get a sense of how they work and what they perceive as big. Still at the end of the day it's a very subjective judgment call system that doesn't offer real guarantees to consumers. If your software needs to be stable, don't upgrade, or spend time to review the changes in the dependencies. The version numbers are no guaranty.