Set up a non standard port. This will stop over night.
It has stopped now (I simply removed port 22 from the whitelist except for connections originating with my home ISP, which is a local ISP where abuse reports actually work)
Have you tried port knocking as a way of temporarily whitelisting the right IPs?