Ask HN: Is Googlebot signing up for accounts on stackoverflow? | Hacker News Reader

Ask HN: Is Googlebot signing up for accounts on stackoverflow?

10 pointschatmasta6 comments

Today I googled a random programming question, and this was the result in google:

http://i.imgur.com/5iRwd1q.png

Note the line "Please click the link in the confirmation email to activate your subscription."

Why would googlebot ever scrape this text, unless it signed up for an account on stackoverflow?

The page in question:

http://askubuntu.com/questions/562417/how-do-you-update-npm-to-the-latest-version

View on HN

Strangely enough, if I Google "install latest version of npm", I get the same link as your screenshot, but the text is as follows:

"8 Answers 8. You can update nodejs by using npm itself, a PPA, or manually. Or just tell it to install the latest stable version. Both may take a while.17 Dec 2014 14.04 - How do you update npm to the latest version? - Ask Ubuntu askubuntu.com/questions/562417/how-do-you-update-npm-to-the-latest-version"

Does that mean that Google have activated the subscription?

It could be possible that they do that, or that SO has granted more permission to the google bot and that string is there by mistake. Good find!

I changed my useragent to Googlebot and Googlebot smartphone and none of them display the link for activating the subscription. Which means that SO had not granted more permissions to Google Bot and possibly Google is registering? Is this even possible?

Maybe this is an error on Stackoverflow's end, (i.e. serving "Please click the link in the confirmation email to activate your subscription." text to googlebot) and then it went into google index.

I'm not sure what you are trying to explain here.

This is new behavior from google. Nobody is aware of googlebot creating accounts and/or logging into websites in order to scrape them.

If this is a new policy, then google set a precedent. Now anybody scraping a website post-login can claim "google does it too," which is already an accepted excuse in defense of scraping in general.

Reply on news.ycombinator.com