There's a fairly comprehensive list at https://reproducible-builds.org/docs/. Debian has a detailed list of reproducibility issues and affected packages at https://tests.reproducible-builds.org/debian/index_issues.ht...
* Non-isolated build environment. This is just asking for all sorts of trouble (users, hostname, network access, etc).
* File system time stamps.
* Recording times in the build process (although even gcc supports SOURCE_DATE_EPOCH since version 7).
* Usage of CPU-specific instructions, e.g. -march=native to GCC.