> When you commit to a version 1, you assume that every user of the package is using every feature you provide through your official API. If you break any slightest piece of the API, you've broken compatibility.
That is not how semver works. If you read the documentation [0], it says right there that the public API must be clearly defined but that can also happen through documentation and you only have to up the major version if you break that specified public API.
You can certainly consider it bad programming practice to expose something via the means of the language that is not defined as public API but it is not against semver to do so.
[0]: https://semver.org/