Hacker News Reader

Scraping may violate of Google's own acceptable use policy, but it's not illegal.

Sure, it's only okay for Google to scrape all websites, but not you scraping it. Got it.

Google is quite scrupulous about robots.txt.

There's plenty of hypocrisy at that company but this doesn't strike me as a case of it.

Scrupulous? Nobody is going to disallow / on Google's user-agent.

I'd suggest respecting Google's robots.txt as much as Google respects our DNT settings.