Legitimately, if you used ex. Ethereum to compile your code could actually prove that a given executable was compiled from given source. Otherwise the only way to verify a build is to build it yourself and check the hashes. Signing just verifies that the build came from a given person, not that they used the source code they say they did.
This was one of the interesting use cases I thought about when I dug into crypto.