aaronwall

>That's a good thing right?<

That really depends on your perspective.

If it were impossible for others to throw trashy links at your site in bulk for cheap it could perhaps be a good thing.

If Google didn't change policies overnight & then retroactively penalize you for things which were fine in the past http://searchengineland.com/guest-post-google-penalty-187707 then it could perhaps be a good thing.

Unfortunately, in reality, both of those ifs are hypothetical & untrue. Which means it is absolutely not a good thing.

>You know Google's ranking system is based on information, and adding more information leads to better results for the Web user.<

Quality of information matters as much as the volume of information. Look at Demand Media's current stock price for an example of this.

But in terms of links, you have to think through the impacts here...

Yes if Google freezes some activities they dislike then perhaps that is a net positive for relevancy, however the more they fear-monger about links the less natural linking goes on. Most of the major social sites put nofollow on almost all external links. And with the sea of fear approach to relevancy, in some cases magazines or newspapers will profile a person & not link to the source in spite of the entire article being about that source. I've spent many hours being cited by journalists where in spite of being good enough to be the primary source for their content, there was no link citation.

paraphrasing David Naylor's excellent recent video http://youtu.be/UdflGrCJxYc?t=1m50s "More and more high profile websites are having a no linking policy. Which really is kind of weird isn't it, because if that is a journalistic website and they are writing about your website surely those are the kinds of links Google would want to see. ... It seems kind of weird that the links Google actually wants you to get are the first links to dry up on the web." - David Naylor

The other important factor is the quality of the information created by those who are trying to do disavows and link removals. When people are irrationally driven by fear while in a harmed state, they are NOT acting rationally. Business owners who are selling off assets, firing employees, stalling with creditors, aligning predatory lending to try to keep things afloat, etc. are stressed out & are likely to make many poor decisions in that rushed & panicked state.

Thus some of that more information which is created is junk misinformation.

Consider this email: http://www.seobook.com/images/link-unremoval.png They asked the website to remove the link claiming it was unnatural, and then they emailed again to ask the website to ignore the removal request.

They don't know which links are "unnatural" and so they are using automation to try to sort through it all. Are their automated quick guesses (which are often driven by tools) more useful than all of Google's internal reviews & ratings data which has been built up for over a decade? Color me skeptical.

There's another factor with the disavow data as well. Look at this removal request http://www.seobook.com/images/trampolines.png Our site has about 20,000 unique linking domains referencing it. However over the years we have had well over a million registered user profiles. If only 2% of the registered user profiles were ignorant spammers who spammed our profile pages and then later added our site to a disavow file, we would have more people voting against our site than we have voting for it. And those profile pages were already not indexed & the links were nofollowed anyhow. Those pages are effectively outside the search game, yet those ignorant spammers can still create negative votes against our site based on the fear-mongering.

And as bad as that sounds, there's no reward for actually removing the bad links either. At some SEO conferences numerous SEO experts have gave the advice to "just disavow them anyhow" even if the links are removed. Thus this deluge of email spam offers publishers no value whatsoever, just sunk cost & wasted time - time which could have been spent creating useful information.

Your thesis would be true if Google didn't have monopoly marketshare & continue buying default (search / browser) placements in other software.

Affiliate marketer & spammer are not natural synonyms.

Some entities might try to say it is six of one & a half dozen of the other, http://www.seobook.com/brand-vs-affiliate-vs-spam but those same entities do not view their own affiliate marketing efforts as spam, even when they are pure spam http://www.seobook.com/google-lends-a-helping-hand

Look at the cloaked affiliate YouTube links to iTunes on music videos. And Google has invested in Viglink, RetailMeNot, etc.

In addition to those types of investments, it was Google's ad programs which funded entities like Mahalo & eHow.

There are zero affiliates which have made as much off affiliate marketing as Google has.

As long as we are concerned about the organic structure of markets...

How do you feel about Google sponsoring educational seminars for market regulators where they do not disclose their sponsorship, actively conceal it, and request the entity putting on the sponsorship do the same thing?

http://www.washingtonpost.com/politics/how-google-is-transfo...

It is also worth mentioning that when Google was lobbying against regulation they indeed DID control & manipulate the placement of favorable media masquerading as regular content:

"the staff and professors at GMU’s law center were in regular contact with Google executives, who supplied them with the company’s arguments against antitrust action and helped them get favorable op-ed pieces published, according to the documents obtained by The Post."

---

Another thing with the line of thinking based on pureness...should companies which have repeatedly been caught rigging actual physical markets (e.g. municipal bonds, California energy, LIBOR, interest rate swaps, forex, etc.) in the real world get a penalty by Google for it?

Or is it reasonable that they take their outsized gains from their market rigging behavior & invest some small portion of it into buying out smaller competitors, lobbying & writing regulations to harm smaller competitors, buy some feel good brand ads & do various charity donations to paint a picture of themselves?

---

I also want to quote the following past statement from a Google executive about the AdWords ads Google puts directly in the search results:

"As with all of our policies, we do not verify what these sites actually do, only what they claim to do."

http://www.washingtonpost.com/politics/google-faces-new-pres...

---

and, some icing on the cake, ... ;)

check out the keyword rich deep links to some of Google's affiliate offers on BeatThatQuote.com (for credit cards, car insurance, etc.)

Google bought BTQ & then put a bunch of cross-site keyword rich deep links on it. is that natural? how is that any different than buying links?

I would bet Matt's IQ is easily far north of 140. He's one of the most perceptive people I've ever met.

Think of some of the stereotypes of engineers. Many can code but struggle to communicate. Matt can code, has his name on many patents, leads a team, makes hundreds or thousands of videos, regularly keynotes at conferences & interacts with thousands of people, was GoogleGuy for years, participates in the comments here, was an expert in some of the past lawsuits against Google ... he has basically had a near infinite number of opportunities to put his foot in his mouth & yet how many times has he ever done it? Almost never, if ever.

About the only times I think he has potentially missed at all was the "breaking their spirits" bit on TWIG, and then two minor bits in a few YouTube videos

http://www.youtube.com/watch?v=muZuX9OaMLo&feature=youtu.be&... "We got less spam and so it looks like people don't like the new algorithms as much."

https://www.youtube.com/watch?v=2jap11IIxfw "the gap between what you can get away with and what google says you can get away with is getting smaller all the time"

---

Who do you know who has as much media exposure as he has had who hasn't repeatedly put their foot in their mouth? For me, that answer is nobody.

---

What makes Matt's performance even more impressive is how scummy some of Google's policies have been, even as he came off smelling like roses. There are endless debates on how reasonable it is for Google to infer intent on links & so on, yet at the same time some of the past Google executive emails have quotes in them like:

http://www.washingtonpost.com/politics/google-faces-new-pres... "As with all of our policies, we do not verify what these sites actually do, only what they claim to do." (for AdWords advertisers)

http://www.theverge.com/2013/1/23/3906310/the-no-hire-paper-... "I would prefer that Omid do it verbally since I don't want to create a paper trail over which we can be sued later? Not sure about this. thanks Eric"

Best of luck staying off of Google properties. It gets more challenging by the day. http://mako.cc/copyrighteous/google-has-most-of-my-email-bec...

Don't like YouTube's copyright claim issues on video game playthroughs? Use Twitch. Oh wait, what's that? Google is trying to buy Twitch. http://kotaku.com/report-youtube-is-buying-twitch-for-1-bill... Oooops.

Links which are nofollowed are still seen by users & can still impact that user's experience with the website. Yet those are fine, even if they point at spam, because they don't have a significant impact on Google.

So then what's the difference between disavows and having the links removed? Getting the links removed costs far more time & money. And it gives Google a data mining stream of feedback they can leverage to dish out further penalties.

Google requires some of the links to be removed not because it improves the web, but rather their goals & interests are aligned with punishment. They want to add cost & uncertainty to SEO in order to discourage investment in SEO from entities not formally connected to Google.

When monopolies distort markets, the impact of those distortions have outsized impacts on the rest of the ecosystem.

Ask Google engineers how they feel about centralization of power in telecom or finance or politics & they will tell you it is the worst thing ever http://www.youtube.com/watch?v=Ik1AK56FtVc

Yet that same sort of centralization is fine when it is Google.

Those same engineers will admit that any algorithm has some level of false positives & false negatives. They may try to minimize these, but they can't make them zero.

The other thing which gets very little coverage, but is crucially important is that in spite of already having a monopoly market position Google keeps buying search marketshare with: their secret Android partner contracts, Firefox default search placement, iOS default search placement, Flash embedded into Chrome (so security updates happen in the background without sending you to a download site where trash can be bundled in with the update) & Flash security updates that hit all other web browsers coming bundled with Chrome bundleware which sets Chrome as your default browser, etc.

If Google decides they don't like you (for any reason), then you need searchers to use a different search engine in a browser which isn't paid off by Google for default placement. And you need to hope that users are savvy enough to repeatedly say "no thank you" to the automatically bundled Chrome install with their frequent Flash updates.

Google mentions how anything is only a click away or similar, but most people tend to use defaults.

When someone changes those defaults on Chrome users, that's a horrible user experience & the "hijacked" settings must be reset. http://chrome.blogspot.com/2014/01/clean-up-your-hijacked-se... But when those defaults are paid for by Google or changed through Google bundleware it is "a great user experience."

If Google believes their marketshare is a reflection of their superior search offering, they are welcome to stop buying default search placement in other web browsers & stop bundling Chrome installs on Flash security updates. But currently they are spending north of a billion Dollars a year on these activities, which indicates they clearly feel there is significant value to them.

The cost of building a quality website & promoting it can easily hit deep into 6 or 7 figures.

The cost of throwing scummy links at a website is under $100.

And if their first $5, $10, or $20 test doesn't work? Then so what for them. They put it on cron job and throw another $20 at it again and again. http://www.seobook.com/images/sape-sape-sape.gif

The key point here is that there's a 100X or 1000X gap between 4 figures & 6 to 7 figures. The cost of burning things to a crisp is much lower than building them up. Whenever incentive structures have a 1000-fold difference in outcomes the path of least resistance becomes a popular path.

>prefer psychology over technology<

yup.

>Nearly everything that's come out of Matt Cutt's mouth in the last 18 months or so has been a scare tactic.<

the bizarre thing is the gap in perceptions internal to Google versus external. https://twitter.com/mattcutts/status/449008706655510528 https://polldaddy.com/poll/7916680/

>IMO all this does is further encourage the development of "churn and burn" websites from blackhats who have being penalized in their business plan.<

Absolutely. churn & burn sites, and then some mixed in parasitic hosting on sites which are already highly trusted.

Even if the sites are not relevant, the existence of a link does not mean the site which the link points at was responsible for creating the link. In some cases (especially with website hacking) those who create the spam will link to innocent third parties to create a smokescreen. Google can & does burn some of these "drive by shooting" sites to a crisp.

Check the people who try to drop profile links & comment spam links on your blog and such. Are there any websites which have as many spammy inbound links as YouTube does? How many people does Google allocate to cleaning up YouTube's spammy link profile?

There's another factor in that Google may shift what is considered reasonable over time. A buddy of mine got an unnatural link warning where the link cited was a link that has been in place LONGER THAN GOOGLE HAS EXISTED.

Now consider that some websites are bought & sold, change ownership, etc. ... does it make sense to penalize today for something which happened 10 or 15 years ago?

Another factor is that just by ranking you will pick up some scraper site inbound links you do not want. Many of those sites in various forms or fashion are monetized via Google ads. Google continues to run their ads on many of those aggregation sites, yet you may get an unnatural link penalty for having links from the same sites.

Good enough for Google, NOT good enough for you TM

Do they (along with the personalization aspects of the relevancy algorithms) boost aggregate awareness & CTR - which in turn further drive rank?

A look at Ask.com's history on a tool like SEMrush makes this line of thinking rather compelling.

Funny you should mention TripAdvisor. Go back about 9 years and guess who was one of the biggest SEO link buyers?

"I doubt any of the much smaller niche/hobby or family business type sites I know about would suffer greatly if Google vanished tomorrow either."

Many existing businesses already have awareness built through a variety of channels over a number of years, but if search were to die off some newer sites struggling to build exposure from scratch would struggle harder. That said, as Google keeps lifting the cost barrier to entry & radiates greater risk out to the smaller players perhaps building initially from the perception that Google is perpetually late / behind the curve on new sites, & that one needs exposure elsewhere isn't a bad thing.

"They are often beaten on volume by at least one order of magnitude by a single link from an influential blogger in a relevant field who is genuinely suggesting that their readers visit that site, or the equivalent via Twitter, Facebook, Reddit, Pinterest, etc. And that's before you count the viral effect"

If they don't have an SEO strategy (or are selling a unique product that people don't know exists until they see it) & just let whatever search traffic happen on its own then sure it is easy for other sources to drive vastly more traffic than search. But if they are focused on commercial keywords, build content around the consumer demands their solutions help solve, and have an SEO strategy then search is typically in the top few referrers in terms of revenue driven.

A lot of the top affiliate links on Twitter or such are for iTunes song sales: low cost, low friction & low commitment purchases. I think I have had maybe a couple Twitter conversions in years, while search has driven orders of magnitude more. But both are channels that are part of the awareness process & when you add them together its more like 2+2=5.

"their foundation is still basically the idea that you can judge the worth of a page by how many incoming links it sees"

I think they fold in more usage data than they let on, but feel little need/incentive to mention that aspect of the algorithm.

"I'm reasonably sure that unless they have some shady deals going on with various popular social media sites, they don't even know about probably 90% of the incoming links to most of those little sites I mentioned"

Did you see the bit about using Google Chrome's data saving caching feature on Android? Chrome desktop also has security features baked into it. And there are boatloads of signed in user accounts tied to both of these. And large chunks of many social sites are crawlable.

"All they've got left is the public blogs and the assorted SEO tricksters, and that's not where a lot of people share genuine recommendations any more."

The fact that those votes are harder to give means they (often) have greater discrimination value. It is easy to say that social signals should replace links, but that would effectively be Google subsidizing competing ad networks. And even if we ignored that the "relevancy" signals in social are also the ad units, outside of those official ads there are third party strategies to ensure social is every bit as gamed as the link graph is.

Just today I saw this http://arstechnica.com/gaming/2014/01/stealth-marketing-micr... and that reminded me of this from the past http://www.seobook.com/images/tweet-and-get.png

Great public relations campaigns make people think they are reading the news without even thinking of public relations. http://www.paulgraham.com/submarine.html

SEO is sort of the same way in terms of there being a range of options. Some people might use SAPE, Xrummer or Fiverr or such (nude person with a URL streaking at a sporting event), whereas others might use more nuanced strategies where any SEO impact appears incidental.

Design itself can play a big roll in the perception of spam. Designs that look crisp can contain nonsensical content without feeling as spammy as they are.

But another factor here is that Google wants to keep raising the bar over time. Things that are white hat fade to gray then black as they become more popular & widespread. And as Google scrapes-n-displaces a wider array of content types, that forces publishers to go deeper (if they can do so without bankrupting themselves).

A couple issues with this:

many topics tend to bleed across niches

sites themselves will change purposes over time

how sites are monetized (and other user experience choices) may change over time

people buy and sell sites

sometimes the desired information is only accessible on sites which are not particularly popular because they are not aligned with moneyed interests

sometimes sites that are popular might be popular because they are inaccurate, conforming to an expected + desired bias

if the whitelist is black and white, entities may change their approach after being well trusted (one of the elegant aspects of Panda is how it can remeasure over time)

Commercial intent is more important than how broad / generic the term is.

A query like "sports" or "news" may or may not have an ad in the search results. Keywords like "credit cards for people with poor credit scores" will almost always show lots of ads, even though the query is quite specific and contains 8 words.

In some particularly valuable verticals (like hotels) Google then further adds their other paid verticals to the results in addition to the AdWords ads.

One of Matt's videos (not sure which one) mentioned online publishers tend to focus a lot on say 10% to 20% of highly commercial terms, while often paying much less attention to the rest of the searches as the commercial terms typically are far more valuable than the informational ones. (Mesothelioma lawyers can afford to pay more for traffic than say people selling flour, or people offering recipes that contain flour).

It looks as brand related searches were not that significant until the Google penalty happened.

http://www.google.com/trends/explore#q=rapgenius%2C%20rap%20...

According to Compete.com about 5% of their searches were brand-related queries https://app.compete.com/website/rapgenius.com/

SimilarWeb also shows most their top queries were unbranded http://www.similarweb.com/website/rapgenius.com

Alexa puts the branded-traffic number closer to 2% http://www.alexa.com/siteinfo/rapgenius.com

(All these tools have errors, but sites like Amazon.com or eBay are more like 15% to 25%+ on the branded traffic front http://www.alexa.com/siteinfo/amazon.com http://www.alexa.com/siteinfo/ebay.com ).

Another way to look at it is if Google detonates start ups from VCs they co-invest with, will their own internal venture arm get squeezed out of some deals?

And if implicit or explicit risk protection (and back channeled information about potential upcoming risks) comes as a fringe benefit of direct investment, doesn't that "free insurance" allow Google Ventures to guarantee themselves more favorable investing terms?

So does this mean when Google talks about these same topics & uses the same sort of black-and-white framing (while encouraging "feeble minded" spam reporting and so on) we should consider them to be intentionally lying?

If no, please explain.

If yes, an important follow up on that front: how dystopian is it that the company in charge of organizing information requires spreading misinformation to succeed?

This is already happening to some degree, however it has generally mostly been the smaller & weaker players who have been soaked first. And in many cases the web & search ultimately helped build up a lot of these players before later knocking them down.

A lot of people are quite apathetic until the impact is direct. They recognize their own usage of the service rather than empathizing with a story from someone who was soaked. Here is a perfect example from a smart, tech savvy person who is big on self-interest / own vertical, but entirely apathetic toward other verticals - and this post is less than 4 hours old https://twitter.com/blam/status/419537614014672897

The tricky part in all of this is that one man's life work is another man's spam. Everything at scale has some mix of original editorial vs user generated content vs scraped & reformatted content vs policing of the mix. Ultimately the entity with a built in audience will be able to win more and more verticals over time due to preferential placement. A lot of people are unaware of how the knowledge graph will keep extending to swallow a larger and larger portion of the overall query stream. Anything of significant value that can be structured eventually will.

One of the more alarming articles I have ever read was one from 2007 where the New York Times explained about Query Deserves Freshness coming about after Google found that their own new finance service didn't rank as well as it should have. That was the beginning of the end for the indy web. http://www.nytimes.com/2007/06/03/business/yourmoney/03googl...

The 2009 recession was the icing on the cake that cut Google's revenue growth rate & got them fired up on moving beyond direct marketing to focus more on brand, brand, brand.

"in 2003 when Denise Griffin, the person in charge of Google’s small customer-support team, asked Page for a larger staff. Instead, he told her that the whole idea of customer support was ridiculous. Rather than assuming the unscalable task of answering users one by one, Page said, Google should enable users to answer one another’s questions." http://www.wired.com/magazine/2011/03/mf_larrypage/3/

Nice try.

Removing 200 links out of nearly 200,000 links in a holiday week is not a lot of work. In fact, Google has told others waiting to be reconsidered that the wait time is multiple weeks. https://twitter.com/Marie_Haynes/statuses/418734227228606464

Further, some sites which submitted reconsideration requests before RapGenius was penalized have not received a response. And in many such cases those other sites:

had far fewer links, removed a far greater quantity of links, disavowed a far greater quantity of links & were involved in something that was nowhere near as overt/intentional on the behalf of the webmaster (I've seen cases where the notices came after a payday loan hacker mixed in someone else's sites with their own links to create cover for their work)

Didn't Google do this same sort of "recycling built trust in channel" when they bought Frommer's, gutted it, sold it back to the original owner, BUT kept the social media accounts Frommer's built up (with the old followers) and rebranded those followers into a new Google-owned Zagat Twitter account?

How is that behavior any less "bait-n-switch" (or influenced by money) than buying a site or renting some links?

So the competitor who paid the SEO is a spammer, the person who paid Google AdWords is an angel & the person who has no exposure has no traffic nor revenues, but is virtuous with great content (that almost nobody cares about other than the owner)?

The hard part about building a sustainable publishing business model online that avoids the largest single channel is that margins matter & if you have 5% margins or such, a competitor that clones your model AND is in Google's good graces can use their search-driven profits to noise up other channels and drive your margins into negative territory. (In social media competitors can target ads at just your followers, competitors can buy your branded keywords in Google, etc. ... most of the big platforms sell access across that way.)

It is not uncommon to see market participants take profits from one line of business or one channel & use it to undermine competitors who were succeeding in other channels.

So lets say you avoid search entirely...how do you sustainably compete against a competitor who has a similar footprint to you AND is leveraging search-driven profits to come after you?

What's correlated?

Are you saying that pages with great content typically have few spammy links? If so, the jury is out on that thesis, as many of the most popular sites have hundreds to thousands of spam sites that scrape just about everything the do & many of those scraper spam sites do link to the original source. And just by ranking for valuable keywords, over time you will likely get some spammy inbound links from sites that are scraping the Google search results.

>>If you're shopping for something, the paid and Google shopping results are often more relevant than the organic results since a lot of them incorporate real time price feeds or promotion codes.<<

In many cases such feed prices are inaccurate. I have used Google shopping many times looking for the advertised price only to find it's "out of stock" or the promotion had ended by the time you click the link. People do the same crap to the other shopping search engines too.

>Advertisers do not bid on keywords that deliver no economic value so it's in their best interest to only show up when they are the most relevant to your query.<

In some case the economic value isn't in the best interest of users. A lot of spam targets installing malware, invading privacy, or similar. Further, there are ads which advise you to "buy x thousand links for cheap" or similar, which at their core suggest you violate Google's search guidelines. And Google had like 50,000 advertisers pushing counterfeit goods at one point (according to Google) and that issue got cleaned up only after Google was hit with a sting operation where they were caught selling ads for steroids for a person posing as a Mexican drug lord. Apparently the drug lord had a high quality score at the time. ;)

In some cases Google has reduced the organic search results to 7 listings per page. On some of the search results where Google has done that they have also added 3 AdWords ads at the top & the bottom of the organic search results.

The reason the prefer ads in the left column to the right is that is where most the attention goes & most of the clicks go. Ads at the top of the left column of the search results might get a 20% to 30% clickthrough rate (higher than that even on navigational search results...more like 90%+). And ads in the right column are lucky to get even a 1% CTR.

>>The difference between Geico, Progressive, All State et al is just the ad price and CTR, in sum how much money you make from them. The best site for the user would be a comparison site or maybe a CNN Fortune article that compared them.<<

Absolutely. And it is worth noting that recent Progressive case, where they defended a person who killed a Progressive auto insurance client (to try to skip out on paying the settlement to their customer's family).

Having a diversity of the same types of companies (basically what often amounts to white label brands with the same exact business model) isn't the same thing as offering various entry points into various stages of the funnel & different editorial opinions.

Also the brand bias in the organic results often makes them parallel the ads once more. And that awareness is often driven by advertising. The day after the Progressive fiasco I remember going to YouTube and seeing a huge Progressive ad. ;)