HN
Hacker News
Top
New
Best
Ask
Show
Jobs
Comment by DataDaemon | Hacker News Reader
Parent
Full thread
DataDaemon
·
Soon, quick and dirty solution: User-Agent: * Disallow: /
View on HN
cr125rider
·
So then the internet police know to zoom up the fiber line and arrest the user-agent for not following robots.txt!
Reply on news.ycombinator.com