HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by scarface_74 | Hacker News Reader
Parent
Full thread
scarface_74
·
Did Google say they aren’t respecting robots.txt files?
View on HN
palata
·
Does anyone respect robots.txt files?
scarface_74
·
Any history of Google not respecting it?
tteggel
·
Google does not respect robots.txt if a page is linked to from other sites
https://developers.google.com/search/docs/crawling-indexing/...
djmips
·
That doesn't say the content but only that it is indexed.
Reply on news.ycombinator.com