> ruining the internet with all this damn parasitic crawling

I feel like we may be conflating the performance implications of crawling with the intellectual property ones.