1. whitelist the finite IP ranges for the regional ISPs/country where you do business
2. blacklist the proxy and tor exit nodes
3. blacklist the list of published compromised servers
4. add spamhaus blacklists
5. add fail2ban rules to trip on common server security scans, and unused common service ports
6. publicly reply to those having access issues, and imply they have bad neighbors.
This will often take care of 99% of the nuisance traffic, but I still recommend live monitoring traffic regularly. ;)