Google undeleted whocalled.us after a front page article on Hacker News

I saw your previous post and could not find egregious blackhat SEO stuff. The backlink profile was not too healthy though, with around 250 domains responsible for over 7000 backlinks. You said you got a spam warning on Google Webmaster Tools: These penalties are hardly ever permanent.

> As usual, if you make a public scene, a large corporation will often notice, and look closer at that individual's issue. I think that's what happened in this case.

> I made 8 reconsideration requests .. all of them were denied.

Probably because you did not show a good will effort to clean up your act. At least you do not tell us what were the contents of these reconsideration requests. For all we know it was: "Reconsider me!"

From your previous post I gather that you were heavily annoyed with the unnatural links actions and warnings. Instead of cleaning up your act for Google you focused on a suggested unnatural link to claim it was all BS.

> As usual, if you make a public scene, a large corporation will often notice, and look closer at that individual's issue. I think that's what happened in this case.

I do think that the people at Google notice a prominent post on HackerNews. But I do not think that they manually removed a spam penalty to avoid bad publicity. If anything: allowing spammy sites in the index without even a slap on the wrist is not good for publicity at all (and very visible to the users). I think the pre-set penalty period was over, or a recrawl found a cleaner site.

> But this experience showed me that Google has more power over my website than I do.

Google has more power over their search engine index than you do. You are free to do whatever you want to do with your website.

I do have a question: The DuckDuckGo widget you had on your site (now changed to a general link), was that an official widget? Did money exchange hands for placing such a widget/link on every page of your site? I am wondering why a site-wide link to a commercial page is not 'nofollow'-ed, and if that may have brought you unwanted quality guidelines attention.

A lot of the "spamminess" comes from practices that were perfectly acceptable back when Google came out, or which are encouraged by the standards which define how the Web operates. For example, each phone number in his database has a specific URL which you can use to access it. Many of these pages have similar features, being that the whole point of the site is to aggregate discussions about suspect phone numbers. This is a similar profile to the "download" sites that have proliferated, except that in this case each URL links to something useful, where the URLs on the download sites link to nothing useful.

Further, the whole point of the Web is to link pages together. That Google penalizes one site author for what other site authors do is a problem with Google, not a problem with the author. In the previous posting he indicated that Google asked him to contact the various site authors and ask them to remove their links. Why should he have to do this?

"In the previous posting he indicated that Google asked him to contact the various site authors and ask them to remove their links. Why should he have to do this?"

He shouldn't have to do that. It's incompetence on Google's part. The solution to spammy links, is to drop their value to zero, not to generate a penalty for the site that is the target of said spammy links. This is a flaw in how Google has chosen to handle low quality sites and spammy links. The incompetence aspect is that they failed to be creative enough to find a better solution, and fell back to using a hammer approach.

This used to be the "solution" to spammy links: Drop their value to zero. This caused an entire eco-system of spammers with the mentality: "I'll just generate 1000 links without the prospect of a penalty in the hope that 10 links stick. I then bill my client for a 1000 links, since there is no way to see which links are devalued".

Google's creative solution was to make sure to webmasters that: yes, these links do hurt your rankings. Instead of linkbuilding and diluting the precious link-signal, these spammers now offered link-profile-cleaning services (Mission. F-cking. Accomplished. http://xkcd.com/810/ ).

So wait, if you want to take somebody else's website off Google, all you have to do is make a bunch of spammy links to it?

This is insane.

Spammers should now be linking to a site and then telling the site owners, "If you want to get your Google rank back, pay us $1000 to take our links down."

This is a thing, in case you're wondering. At my current job, there was a spike a week ago with more than 1k spam links. We've no idea where they're from, or how to disassociate ourselves from it

It takes some effort, but you can disavow those links:

https://support.google.com/webmasters/answer/2648487?hl=en

You can disavow links via webmaster tools. The onus should not be on the innocent site however, that's bad engineering design by Google.

Yes, and the solution is to penalize the sites guilty of sending out the spam links, not to penalize the receiving sites.

To penalize Good Site A, because Bad Sites B/C/D/E are sending spammy links to it, is an inherently morally bankrupt premise.

The web has changed and is still changing into a more dynamic app-like structure, from static content. With that come new guidelines in regards to (search engine) accessibility. Google also is battling spammers. Spammers can saturate a single technique (such as guest blogging) to the point where Google says: "We are actively making sure that this technique does not benefit you in the rankings".

It is not the site structure that is entirely at fault here. It would not make sense to aggregate phone numbers and discussions about these numbers on a single page. A "thin content" penalty is speculative and Google even gave an "unnatural links" warning, telling us where to look first.

I think Google understands the power of linking pages together and weighing these links in ranking :). Reverse SEO is murky though, and in principle I could agree that other webmasters devaluing your property is a problem with Google. On the other hand: The fact that spammers complained a lot (and still complain) and were being re-purposed by Google to help clean up the web, tells me that Google is winning this battle. But we also know that collateral damage is possible in war.

Finally: Google does make mistakes. And a mistake is between algorithmic superiority and malicious intent by Google.

> In the previous posting he indicated that Google asked him to contact the various site authors and ask them to remove their links. Why should he have to do this?

Because else he will be slapped with an unnatural links penalty, and he clearly doesn't want that. Again, he is free to ignore these warnings and see his site removed from Google's index. Google can not force him to do this. They can however enforce quality on their own turf.

Question: Could you clarify some of the currrently spammy practices which are encouraged by the standards which define how the Web operates?

I admit my complete ignorance in this matter. I do not understand why this would induce them to remove their links.

They do not seem to have anything to gain, and actually, it sounds like it might be highly gameable as a form of negative SEO.

At best, it seems to me that there would be an inverse ratio between the likelihood of the link being removed and the spamminess of the linking site.

If someone could explain this, I would be grateful.

[alternately, I'll delete to reduce clutter if I get bumped down some more.]

> You are free to do whatever you want to do with your website.

When you are dealing with a monopoly, which Google has on many large segments of traffic type (i.e., Bing and Yahoo are non-relevant in those segments), you are at the mercy of that monopoly. And hence they have the power over you. Which is often abused by being dismissive towards you.

Google keeps forgetting that the website owners provide the content that Google indexes and monetizes-on to sells ads (via search and other channels), and that they are in a relationship.

Except that relationship is often abused by treating webmasters like they are garbage.

When you rely on Google for visitors/income, then they have some power over you. Agreed. Google still can't force you to change your website content. If you run Adsense on your site, and Google finds that your website content quality is low, then they will simply stop their contract with you.

That Google abuses their monopoly to be dismissive towards webmasters and treating webmasters like they are garbage is, as it stands, unfounded. Which other search engine has an active webmaster forum, webmaster education and webmaster tools? Which other search engine warns you about spam or technical problems on your sites?

> Google keeps forgetting that the website owners provide the content that Google indexes and monetizes-on to sells ads

Google does not forget this. It just hails quality over making a buck, shown by banning ads on websites that do not conform to the quality guidelines.

I was not referring to anything adsense specific.

On the rest, if you believe that, then you must have been absent for the other million times when an individual was thrown into the garbage bin by Google, and told "you did something wrong (maybe), but we won't tell you what, nor will we ever respond to you again."

> It just hails quality over making a buck, shown by banning ads on websites that do not conform to the quality guidelines.

While this is not true for every category, in my personal experience and in many reported other cases that analysed the traffic sent by Adwords' "partner network" (that Google derives a large part of their profits from) is half made up of auto-generated sites, links farms, and automated fraud clicks, and has been so for the last decade. This, and other shenanigans (like the 500MM pill ad settlement) goes against what you are claiming.

> Which other search engine has an active webmaster forum, webmaster education and webmaster tools? Which other search engine warns you about spam or technical problems on your sites?

Bing has many of those.

> On the rest, if you believe that, then you must have been absent for the other million times when an individual was thrown into the garbage bin by Google, and told "you did something wrong (maybe), but we won't tell you what, nor will we ever respond to you again."

I've seen this a lot (too much). In fact this attitude of "Google sandboxed us and we have no clue why!" is as old as SEO forums. I've helped people fix their on-page and backlink issues. I've submitted successful reconsideration requests. I've made ads cheaper by linking them to higher quality pages. I did not fix this by picking them out of some hypothetical garbage bin. I fixed this by fixing their honest and dishonest mistakes and making them compliant with the quality guidelines.

Here at HN we have hellbanning. Do people get hellbanned by mistake (probably, but hopefully rare). Should we respond to every single person hellbanned, and explain to them what they did wrong, while they continue to be oblivious haters? It is a matter of how you view it I suppose.

I do not think that half-made-up fact is true, but I will not challenge you on this.

What was going on with their link profile? The other post had no comments that talked about specific backlinks.

The original post made it sound like there was no act to clean up. If there was spam link building on their part, it changes the story.

I am not in a position to give a full report on the quality of the backlink profile.

> The original post made it sound like there was no act to clean up.

That seems standard: from innocent to the most hardcore spammer, of course they didn't do anything wrong, Google made a mistake and abused its power.

> If there was spam link building on their part, it changes the story.

It could change this story. But I do not want to point any fingers, and I think Google is smart not to reply on this case (which is kind of a shame. I miss Matt Cutts and I miss the time when HN allowed for these open discussions. It just seems too much of a PR risk these days.)

The term "backlink" isn't part of my vocabulary. I ignore Google ranking or external linking and focus on building useful or interesting things that adhere to my principles.

There was nothing to clean. Lots of websites link to whocalled.us, but I don't pay attention to them, and I certainly had nothing to do with any links to the site.

"Pure Spam If you see this message on the Manual Actions page, it means that Google has detected that some of your pages may be using techniques that are outside our Webmaster Guidelines. The site appears to use aggressive spam techniques such as automatically generated gibberish, cloaking, scraping content from other websites, and/or other repeated or egregious violations of Google’s quality guidelines."

My reconsideration request was usually, "You're idiots."

A wise person once said to me, "If you don't like eating shit, why did you order the shit sandwich?".

I had a sandwich shop, and the city put up a wall blocking access to the driveway from the main road. They put a pamphlet in my mailbox that said, "Pure Shit," which instructed me to stop doing things like grinding up cats, posting obnoxious fliers on everyone's car windows, and stealing tomatoes out of people's yards.

There was a little box I could write in and mail back to request they restore access from the road. I made sure the shop wasn't doing any of those things, I cleaned it up a bit, and spoke with the Mayor. He said he didn't know why people would want to eat sandwiches, and my shop sucked. My form was mailed back with a "Denied" stamp and the same generic paper about how my shop is "Pure Shit."

"You're idiots." is a reasonable response of someone forced into a situation where their words have no effect, and nobody appears to be reading them. Why waste effort reasoning to those who refuse to listen or acknowledge what you say? Nobody said, "Your tomatoes are tainted because your supplier is a known fraudster." It just said, "Pure Shit."

Do they even have official widgets or affiliate programs? I linked to them because I wanted to share the link with people.

Fixing mistakes without accepting blame or acknowledging the error always bothers me. Does it stem from the fear of a lawsuit-happy society, or embarrassment?

I suspect that a lawsuit-happy society is a good part of it.

Having been on the other side a few times (the one who quietly fixed an error without acknowledgement), there are a number of other factors at work as well:

1. Sometimes, fixing the error is within my power but talking to the customer is not. For example, when I was at Google, I was an engineer - fixing a bug was something I could do merely with a code review, but talking to the outside world was forbidden by company policy, and I didn't even know who the people were that were authorized to speak to the outside world on company behalf.

2. Very often, the error is one of several dozen in a bug queue, and you just want to fix it and move on. Talking to someone - particularly a customer - is a big mental context switch; it means that 5 other people won't get their errors fixed at all.

3. Sometimes the error is a distraction from your normal job duties, you feel like you can do a good deed by taking 5 minutes of your day to tweak a config file, but you don't want it to blow up any larger than that, and usually once a user has your attention they feel free to ask you to fix all their other annoyances. This is a big problem with working on open-source software; I've never yet had a good deed take less than 5x the amount of time I budgeted for it, and usually involve a few people mad at me. (I often find out later that many other people are very appreciative, but the latter never seem to speak up).

I think the fundamental problem is a mismatch between our expectations as human beings, which are shaped by millenia of face-to-face interactions, and the reality of modern society, where we are just a ticket number to a faceless corporation. Internet companies like Google and Facebook have roughly a million users per engineer working on the product; Whatsapp has 50 million. Imagine the scale of insignificance of any one problem; by the numbers, a user of a consumer product is roughly as consequential as a hair on your head. That could be why we get such shitty service from basically every major consumer company out there.

"I suspect that a lawsuit-happy society is a good part of it."

In this particular case and specifically with respect to google (and their resources and profitability and for lack of a better way to put it "power") as well as how difficult it would be for those who are wronged to bring a lawsuit (and prove damages) I somehow strongly doubt that has anything to do with it.

The only thing that I think that google would fear is regulatory action (if even possible) or perhaps (once again if possible) a class action lawsuit.

This seems likely:

"fixing the error is within my power but talking to the customer is not."

To be fair, "a temporary ban expired automatically" seems equally plausible...

The name appearing on the front page of HN may have changed how many people were searching for the site and clicking its link, taking it from the spam bucket to the content bucket.

It may have also changed google's view on which people are searching for and visiting the site. Before, its search traffic was people searching for phone numbers, a random cross section of people who get spam phone calls. After, its a specific name brand being searched for by possibly well connected tech workers who are logged into google+.

Fair points, yeah. I think this is akin to fitting curves to a single data point.

> I suspect that a lawsuit-happy society is a good part of it.

For the most part, the people complaining about lawsuits have something to lose. I think there's a reason for that. Lawsuits are the first resort when the track record of the people with power is such that trying to convince them to act ethically does not seem plausible. If problems got resolved without having to go to the mat with powerful entities much, much bigger and with many more resources than you have, you really think this'd be a thing?

There's a possible case to be made that Google should be regulated in some way, perhaps as a public utility, with requirements for human interaction. It's unacceptable that they have as much power over the Internet as they do and still hide behind the facade of a machine.

You might make a case for regulation if a) you could show that Google is a monopoly and b) Google used its monopoly power to shut out competition or control market pricing. But, at least in the U.S., the former is not true - a market share of 75% is necessary to establish a per se monopoly - unless the latter is true, and the latter is patently untrue. "Attract a lot of eyeballs" is not, thank goodness, grounds for regulation, or Twitter and Facebook would surely be regulated.

There is no facade. Google search is algorithmically driven, and no one has ever shown that its algorithms discriminate unfairly.

Your experience is scary-pleasant. Perhaps someone should start a peer-reviewed list : why-google-blacklisted.me where the top voted ones would stand a chance of getting reviewed by non-bots at Google. In all likelihood, a situation like this gives nightmares to entrepreneurs on a daily basis.

Edit: Another recent similar incident: https://news.ycombinator.com/item?id=8873149

So in essence you want thousands of people to volunteer their time and energy to do the work that a multi-billion dollar corporation should be doing in the first place?

"Don't be evil" at this point is a punch line

You didn't give an alternate solution?

Someone mentioned here there's a million to 1 user to engineer ratio. As much as I would want Google to fix the issue, I don't expect that happening anytime soon. If all of us can collectively help the community fix bugs then why not? Just like countless other crowdsourced intelligence tools (e.g. wot browser extension, ad blockers etc.).

Isn't it obvious?

The alternative solution is that google takes responsibility for the quality of their search results and actually pays people to work on it. I know they like to pretend that everything is automated but we all know it clearly isn't.

And no, they don't need to hire a $300,000 a year engineer to do this work. You don't need advanced programming knowledge or experience to be able to judge whether a website has a real case when they submit a reconsideration request.

It could also be that someone from Google saw the post and put in a request for reconsideration on your behalf too. Unfortunately, a perk of working at Google means that any requests by Google employees get extra consideration (at least with several issues friends have had to deal with involving Google taking down videos/channels on YouTube)

Isn't it likely that they just made a minor "algorithm" update that fixed the real reason why your site was penalized?

Maybe they reset your penalty as a "thank you" for bringing it to their attention?

Google search is FAR more human-driven than they'll ever admit to - for legal and PR reasons.

I think most people who are even remotely interested in SEO know this to be true.

Or remotely familiar with modern machine learning. With a company of that budget, all important systems ultimately go through a supervised learning layer. Practically nothing important uses unsupervised learning end-to-end.

I am the cofounder of a company that serves as an alternative to AdSense. Get in touch with me: joe@publir.com.

what does this have to do with adsense or its competition?

OP gave up on AdSense.