It depends. I've seen "oh well if someone has rce they probably have root anyway" used way too many times as an excuse to avoid defense-in-depth measures.
ASLR, NX, and CFI would be an example of a defense in depth stack that is meaningful.
SSH, Fail2Ban, and SPA would be an example of a defense in depth stack that basically just wastes time.
I would be more comfortable with a system where I knew I had to burn the box if I lost RCE on it than I would be with a system that somehow depended on RCE not coughing up kernel, and persistence, to an attacker.
The other thing defense in depth can provide is increased attacker cost. That's why there are economically valuable DRM systems (BluRay's BD+ is an example here). All you have to do is push attacker cost across a threshold (for instance with BD+, that's keeping titles secure past the new release window) to make a defense in depth control valuable.
But if someone has a kernel exploit, probably nothing you've done for defense in depth is going to meaningfully increase costs.
A really good example of this is Spyro 3: The developers set up a system of overlapping checksums (which could in turn bet part of the data being checksummed by other, overlapping, checksums) so that it was virtually impossible to change even a single bit without failing the test. It was eventually cracked, as the check only ran at boot time (it required 10 seconds of disk access, and adding 10 seconds to every loading screen in the game would have been unacceptable), which meant it took over two months for pirates to get a crack working (unusual for the time). And since most game sales come in the first two months...
But that's really just me using this as an excuse to share a bit of technical trivia.
I agree with you that ASLR, NX, and CFI are the most important system level defenses to employ.
This assertion confuses me.
I use fail2ban on boxes I have key-only ssh configured for.
Are you aware fail2ban works for services other than ssh?
If an attacker / script knocks unsuccessfully on my ssh door, other doors are then closed to them.
I also get much (much!) cleaner logs thanks to fail2ban.
I suspect that you're confusing fail2ban and port-knocking (or using fail2ban as a port-knocker).
The point of fail2ban is to prevent an attacker from brute-forcing your server. In a key-only config, the chances of getting brute forced is smaller (by a few orders of magnitude) than getting hit by an asteroid and having the server get hit by an asteroid, so fail2ban doesn't really help.
_In theory_, the same would be true for port-knocking.
However, in practice, sshd can have security holes which a malicious scanner could exploit. And while port-knocking doesn't help against a determined attacker (it's subject to MITM, replay-attacks), it does help with defense-in-depth.
Too bad that sshd can't enforce use of password-proctected keys on the server side..
My point is that in general it would be better to disable password auth and only use key based auth, but only if you could somehow guarantee that the users wouldn't do crazy things like use password-less keys. But as you can't do that on the server-side, what other options do you have?
It's about reaction of the staff to key leak:
>> A HPC center [...] disabled key logins IIRC due to some incident where an attacker had got hold of a password-less key.
This reaction seems just silly.
There's more to it, but that's the flavor of it.