I've always wondered. How do these guys hit google search so much without hitting limitations?

Google is pretty aggressive at banning bots, and I can't imagine Google have given a competitor API access or something like that. Proxies is out of the question for this scale too.