at a previous job i cleaned up after such mess. they used to have fail2ban adding thousands of rules without ever deleting them automatically. I replaced it with a pam module that maintained an ipset for addresses with failed login attempts.