Comment by kmontrose | Hacker News Reader

Comment by kmontrose | Hacker News Reader

This was dealt with more than a year ago, when we did remove nofollow from trusted posts links' http://meta.stackoverflow.com/questions/111279/remove-nofoll....

Would have been dealt with even earlier, but our experiments with it (http://meta.stackoverflow.com/a/69032/130213) lead us to believe Google was starting to treat us as a link farm.

What we were seeing in our earlier attempt wasn't pages appearing lower in results (which could reasonably be expected in some cases as a result of the change; and we wouldn't care, 2nd on a search under an equally good resource is fine by us), but pages being removed entirely; almost universally pages that had newly un-nofollow-ed links to stores, or ad-laden sites (mostly legitimate posts, but some spam that stuck around for a bit before the community deleted it). So, "classified as link farm" seemed like the most likely problem. Google naturally won't tell you why your ranking drops, so it was (and remains) just an educated guess as to what happened.

So we stopped, putting it on the "wish we could, but reality doesn't let us"-list until Google reached out to us (among thousands of others, I'm sure) to change our nofollow practices. Google didn't describe any changes in their algorithm, but it seems reasonable to me that there would have been some tweaks around nofollow to accompany a new policy; again just an educated guess.

Basically, this is an old post complaining about a long since addressed concern that we had tried to address even earlier but ran into practical problems with.

While the exact details of our algorithm are secret by necessity, I will say that we've had to consider posts individually to prevent spammers from kiting a single account up to post nofollow-less spam content. People still try, it's kind of astounding how much spammers try (I suspect SEO's opaqueness cuts both ways here), but it doesn't work (well, you can get one link in your profile; but there's less SEO juice to pass and you are hard-capped at one, no matter how long it takes someone to delete your account).

Disclaimer: Stack Exchange employee, I was on all the relevant calls but has been a couple years so grain of salt and all that.

I'm the original author of the article.

I'm not sure why this made HN 2 years after posting it, but I wanted to make a comment.

SO has implemented a way to remove the nofollow links, but it is way too strict, and probably only affects a very very very small percentage of answers. I'd bet less than 0.1%.

For example see this answer with 74 upvotes from a user with almost 100k reputation. The links are to MSDN (which is probably not spam by definition) and to a quoted source on techbubbles.com. http://stackoverflow.com/questions/2660355/net-4-0-has-a-new...

Those should have been un-nofollow-ed (went and looked the algorithm up), so that's probably a bug.

I'll dig into it, should have a fix in the next deploy (assuming it is a bug, but I don't see what else it'd be).

You're trying to decide on your end how many upvotes/downvotes should trigger nofollow - have you talked to Google about a standard (which presumably sites like Reddit could also use) to expose that information so Google's crawlers can reach their own conclusions on a case-by-case basis if need be?

Hm. I took a look at a few of my highly voted comments. Some of them have nofollow, some don't. I can't really tell why. For instance, this answer: http://stackoverflow.com/questions/1990464/efficiency-of-pur... has lots of links in it, citing sources, all of which are nofollowed. A few more of my lower voted answers do not have nofollow. But this one does: http://stackoverflow.com/questions/1946426/html-5-is-it-br-b.... As I go through my history, I find that most of my links are nofollowed; even ones that get more than 25 upvotes, have been around for years, and I'm within the top 1% of reputation site-wide: http://stackoverflow.com/questions/5724522/creating-github-r....

So, either the algorithm is really strict and fairly capricious, or it's quite badly broken.

This was also the exact time of all the Google Panda tweaks and the fallout from Google starting to penalize creative commons scrapers of Stack Overflow content. There were a lot of balls in the air and only a few of them were ours, so we wanted to be careful.

When Google is 90% of your traffic, you REALLY REALLY REALLY do not want to get on their bad side, even accidentally, so I can hope you understand why we wanted to be cautious to the extreme here. If Google decides you're not doing things right, it is literally a business ending move.

Unless your VCs and investors are cool with you losing, y'know, ninety percent of your traffic.

Somebody please remind the community about this post the next time someone says "You only have to care about Google algorithms if you're a fly-by-night spam site."

Of course, if/when there is a SO-penalizing bug in Stack Overflow or Google code, no-one will be able to fix it since they can't Google for Stack Overflow answers! ;)

I've seen this happen to other sites, something angers the Google algo and organic traffic falls off a cliff.

History seems to bear out that SO correctly identified the source of the "problem", and SO is such a valuable resource that I'd rather it be Google-visible over passing pagerank.

This is the problem when you have a single player dominating the search space with an opaque algo and a very limited appeals process.

I think "dealt with" is a strong way to phrase it.

The site still applies rel=nofollow to what seems like the majority of outgoing links. Since I would call the overwhelming majority of content on Stackoverflow not spam, there's a lot of good links getting nothing.

Reply on news.ycombinator.com