ipsets can handle vast numbers of addresses. I'm sure fail2ban uses them these days.