Hacker News Reader

> Why would I ever let bing crawl my site if they aren't going to send any visitors to me?

How can you refuse? The only way I know would be to require an account, but even then they could bypass it.

Major search engines honor robots.txt

If it becomes standard to have such a file and it effects their bottom line, could they disregard it?

If the file that were previously honored as consent to use the copyright material is subsequently ignored, wouldn't the content creators take the indexers to court for copyright infringement?

Reply on news.ycombinator.com