Sure, but I also trust it a hell of a lot more than any port knocking implementation. If SSH is your point of trust... make it your point of trust. Throwing up an extra layer of obscurity on top doesn't help anything.
I don't quite understand why you're saying it adds nothing at all.
You're not defending against the attacker who is targeting you with this. You're defending against the attacker who is targeting "anyone who is trivially accessible."