ParentFull threadTobu·That /robots.txt does tell the other bots to not crawl most of the site. These results are unlikely to stay.View on HN