ParentFull threadthe8472·In addition to crawl-delay there's also HTTP 429 and the retry-after header.https://tools.ietf.org/html/rfc6585#page-3View on HN