"I suppose"? This has been done for years already. There are even GUIs for that (so that people don't have to use the CLI) which tell you "between 1.2.3 and 1.2.4 you did the following: ... This will break compatibility in the following cases: ..."
So it's better to not even try?
Additionally, for many changes it is reasonable and possible to check whether behavior has changed. Not every function has an unlimited number of inputs and outputs.
That's one of the biggest reasons I avoid Go and PHP: It's not only that the developers ignore existing solutions if they haven't "invented" them themselves, but that they don't even care about improving things where no "perfect" solution is available.
Ignorance and arrogance is a combination which doesn't make me trust them, and products based on their software. (OwnCloud, Docker, ...)