Full threadRandomWorker·https://daverupert.com/robots.txtMaybe change this? Simply add:User-agent: * Disallow:To allow all crawlers to the site.View on HN