I feel like this is a great example of why this "obscurity as a security layer" is misleading, to say the least. Having a "great algorithm" that you don't publish is bad for the same reason security by obscurity is bad - there are fewer people criticizing and tempering the algorithms you use. This feeds into the idea that "anyone can make cryptography they can't crack" (NSA or not).
The problem with these "obscurity as a valid security layer" arguments is that there's already obscurity built into these protocols. The algorithm is open, the key (or password) is kept obscure. Adding some non-standard flourishes on top of a secure protocol is basically just using an insecure protocol (security by obscurity) to "extra" secure an already secure protocol. The point being that the overall security of the system is not meaningfully increased by these actions.
That said, I understand why you'd want to use a non-standard SSH port or port knocking or something to reduce noise on your logs.