Full threadNormal_gaussian·"Well-behaved bot - Honors robots.txt directives, including crawl-delay"From the behaviour of our peers, this seems to be the real headline news.View on HN