Bare bones hardened kernel + shim init + target application are all you need, and will be your highest security/reliability systems.
And yeah, threadrippers are a must.
Security comes down to reducing attack surface, ideally to an infinitesimal degree.
Sure, compromised binaries are nasty but personally I do place quite a lot of trust with the distribution repos.
(PS, if you are reading this and contribute packages to distribution repos: Thank you!)
In security focused orgs though you review all code yourself with the exception of things with extensive third party signed review such as the Linux kernel itself. Even then I review codepaths in the kernel critical for my use case such as random.c
From there, if I -alone- compile containers, kernels, or binaries, someone could coerce me to tamper with them to compromise all downstream users. Same if there was a central build system I can access. To mitigate this I ensure my artifact builds are deterministic, sign my changes, and have team members review my changes, reproduce my artifacts bit for bit, then counter-sign the results.
It is never wise to be in a position where there is possibility of you yourself tampering with things that control anything of value, or else someone will coerce you to help them steal said value.
As a security engineer it is my job to ensure no one ever has to trust anyone, including me.
Interesting thread!
I mean, they give a wonderfully higher quality of life for compiling but they are $2,500-6,000 just for the CPU. Of course this can be done on much lower-cost CPUs! A 12-core Ryzen is already at the "I have lots of money to spend" end of what I'd suggest for something like this.