Hacker News Reader

Would adding robots.txt help? I cannot blanket ban the page as it needs to display to the user with 0 or more results and my understanding is that whilst robots.txt would take precedence for index settings as I have it defined in the header for that page it ought to achieve the same.

View on HN

I think adding rule for /search* to your robot is easier way to block google bot

Don't do this until Google sees your new robots meta directives (or canonical tag) on these pages, that way they will drop from the index. Then you can add this to your robots.txt to prevent it being crawled again.

Reply on news.ycombinator.com