Hacker News Reader

Comment by jwatte | Hacker News Reader

My pet theory is that part of the anonymous usage data Chrome sends back, is digested page contents that go into pagerank. And such browser level digesting would be on rendered pages (after JavaScript execution.)

I have no other reason to believe it is true other than it's what I would do to distribute the job of crawling the web to my users if i were Google :-)

View on HN

I think that is unlikely because then SEO specialists would generate and send such data to improve the positions of their websites.

But Google could use this to discover new URLs that are not linked anywhere.

Some URLs are private by design, so I think it would be an awfully bad idea to discover new URLs in this way.

There were cases when such private URLs got into search engines. You should protect such URLs with authorization or at least hide them behind a POST form. Or block them with robots.txt.

Also if a user follows a link from such page, the URL would be leaked via Referer header so it is not secure anyway.

I feel robots.txt is the most effective to prevent a private site from being crawled by Google followed up by an inline <meta name="robots" content="noindex"> tag.

Chrome by default searches Google with whatever you type in the top bar - so by not disabling that private URLs are sent to them and then are scanned (however if the correct robots.txt file is setup on those URLs then Google Crawlbot would stop at that point).

I believe it was this setting (I have it disabled as well as other options - however I believe it's only this option that sends data back) [1].

[1] https://i.snag.gy/cmkGzp.jpg

Yeah, it would make sense. There has to be more reason to giving people a free web browser other than just that it uses Google search.

Chrome does send domains/urls entered in the omnibar to google (I think someone did the experiment to test it several years ago), but sending out page content to google? If it were true it'll cause a huge legal and privacy problem, especially in places with tight privacy law like Europe.

Exactly, imagine if Google sends the page content after you log into your bank account.

And yet this is exactly what happens with automatic translation.

Microsoft did exactly that with their Bing toolbar.

https://www.google.com/search?q=hiybbprqag

The reason is Google's interest in advancing the web platform. They're afraid of content migrating into either native apps or Facebook, and Chrome is their (somewhat successful) attempt to make the open web competitive.

Reply on news.ycombinator.com