And there are roughly 1267650600228229401496703205376 port 22's in the IPv6 space - I've substracted a few for reserved and unassigned spaces, but at this scale a few orders of magnitude hardly matter.

Here, for comparison:

  281474976710656 - total ports in IPv4 space
  18446744073709551616 - 4 port combinations
  1267650600228229401496703205376 - my estimation for 22 in IPv6

And if you don't block the knocking when receiving traffic on another port, brute forcing gets quite a bit easier. I mean, it's still unreasonable. But my point is, when we accept 0,001% chance as possible, I don't think we can say that 0,000001% is impossible - just a lot less possible ;)