A similar problem does exist when hosting your own crawler. No matter how polite your crawler is, after a few days your IP address gets blocked.