1. Make all port not respond (Stealth in the firewall), unless they are public like http..
2. Change the SSH port # (over 8192 also)..
3. Setup port knocking watchdog so they have to knock first in a specific order on three ports before being allow to connect to real port.
4. Setup fail2ban. Including if someone pings the knocking ports (in the incorrect order) or real ssh (Without knocking first) then after a couple of times, add their ip to fail2ban list for 48hours..
You get rid of 99.98% of the lookers instantly, by just doing step 2...
This assume you have control over the server, there are several script online that help you provision something like that with ansible.. (Most of them helper related to configuring fail2ban.