I had inherited a server (Arch linux) which ran something like fail2ban (can't remember what it's called). It slowed down the machine tremendously, because the iptables lists became very big, and every packet started taking up too much CPU. I had to disable it (switching to whitelisting instead). Did you ever encounter something like this with fail2ban?