That's very good hint, thanks.
There's only been a couple of remote ssh exploits (that I'm aware of) and both of them were stopped by white listing. If you can figure out your address ranges, I think it still makes sense to white list. I guess also the bots will catch up with modern ciphers.