The path from source code to distributed binary file is known to be a blind spot.
Removing that blind spot is either Harder Than You'd Think or Easier Than You'd Think, depending on your perspective and expectations. You can find some issues listed here:
https://reproducible-builds.org/docs/
Or the homepage of reproducible-builds.org for a general take on the subject. (I am not associated with that website.)