If you think that way, then everything is incompatible, because the person using your lib could always do a hash of your lib file and create code that depends on it:
if hash(libpath) != predefinedHash {
log.Fatal("broken compatibility teehee")
}
In the context of semver, "API" means "public API". There is no other API but public API.