a: $port != 22 is enough to thwart most bots and skiddies. If you think the port number is a guarantee that you are safe or that you are communicating with a blessed ssh you are sadly mistaken.

b: Uhh the port number means nothing. Host keys are there for a reason... Someone does not understand the functions of SSH. http://www.snailbook.com/ <- great book

c: If you are not investigating fingerprint issues when logging in via SSH and you call yourself a sysadmin, please stop. You are going to be the reason your company ends up in the news because your shit got owned and 2,000,000 user account hashes were leaked blah blah.

d: If you are not using key based auth and you have a fly by night keystore policy. Which means you have a keystore - stop. The whole keystore for SSH shit irritates me. I cannot tell you how many times I have heard sysadmins say a that a single private key is a "best practice". It is not a best practice it is a stupid practice and really prevents you from protecting unauthorized logins on other machines for the obvious reasons.

   Put your public keys on bitbucket.com or source 
   management. Put your private keys on an encrypted disk
   in an encrypted archive if you must. This is still dumb
   imho because it is not needed.

   Leave one account (root) with console only/no ssh access
   that will allow for keys to be revoked/recreated when
   users need new keys.

e: The original article http://www.adayinthelifeof.nl/2012/03/12/why-putting-ssh-on-... Is wrong and misguided. port knocking or knockd is an obscurity measure, precisely the kind he argues against. The linked article from the OP calls this out.

f: Spinning up daemons is a big deal for non-priv users? So spinning up a remotely accessible Lisp out of emacs from a screen that is running in the background is bad? Hmm, here I thought that computers were meant to be tools for humans to get work done... Sorry, background processes are part of getting shit done. Users should be able to spin up the stuff they want to spin up in the network segments they have access to without the bureaucracy of misguided fools making the jobs of others more difficult because they think spinning up a gunicorn process or a custom daemon is worse than their unpatched kernel, apache tomcat and mysql listening on a publicly accessible address. Stateful firewalls and hosts allow/deny are there for a reason.

Sorry for the snarky reply here but there are a lot of people chiming in that obviously have very little knowledge about managing *nix ops and remote access. I have pretty strong opinions about this kind of stuff. Especially the single key stupidity and not checking host fingerprints.