Except when physically logged in via console you're already using ssh before using sudo.
So the complexity you describe is already there.
sudo removed is one less moving part in the end.
So the complexity you describe is already there.
sudo removed is one less moving part in the end.
If you forced me to choose one to remove, I’d delete ssh in many cases. Anything production that isn’t bare-metal is a candidate for never allowing a remote terminal. Easiest with cloud instances since they’re almost completely disposable, but many sites still don’t have the stomach/discipline for it.
Yes, removing sudo is one fewer moving part, but sshd is a much larger moving part than sudo. (If you think sudo is a larger moving part than it should be, I'd agree, and you can use doas instead.)
Regardless, the vast majority of my sudo usage is on my local machine, so there's no sshd involved at all.