Google, Bing, and SERP Copying: Additional Evidence? | Hacker News Reader

Google, Bing, and SERP Copying: Additional Evidence?

hung-truong.com

20 pointshung11 comments

"Otherwise they have a very similar method of cloaking detection, a similar policy for punishment, and a similar timeline for reducing the punishment for cloaking… At this point I feel that there are too many coincidences for this to be mere chance."

I disagree. The realities of having to work in the real world will tend to push Google and Bing in the same directions, even if their development is fully independent. The possibility of independent implementations of similar policies is not so unlikely that it justifies accusations of copying. I think you need more evidence.

Besides, it can't flow along the pathways that were previously established to my satisfaction (YMMV). If I understand it correctly, the Bing toolbar analyzes click patterns to see how pages connect together on the web, Google used the Bing toolbar while making their spiked searches, and found that small amounts of that signal made it into Bing, which was exposed when they created searches by design that had no other signal. I'd lay money the Bing toolbar now has an exclusion for Google (and probably other search sites), if only because Google demonstrated it opens an attack vector from the competition. And I do not think the Bing toolbar could pick up on cloaking like this, because that would rest solely on Google's internal servers. I'm not even sure by what mechanism Bing could be copying this signal from Google.

Agree with jerf. And furthermore, looking at the chart provided its not clear to me that one is trailing the other. For example, Bing has a big dip on March 14th, which is prior to the date that Google does. And even before the March 15th date your incoming data isn't very flat. And during the March 7th week Bing seems to again lead Google on a downward trend.

I don't think you can assume anything from this data at all.

What would be far more useful, to prove this hypothesis, is request to Google to not show the site for a week and see if Bing's results fall off a cliff. I suspect they won't, but give it a shot.

And one more thing... Google seemed to suggest that the data took months to go from Bing toolbar to search results. You seem to suggest it takes one day! That's a pretty big difference in the efficiency of mining their clickstream data.

This is a good point. I am not sure how long it takes for change to propagate, but you are right; one day seems pretty fast.

I also had a suggestion to show different content to Goooglebot and Bingbot. While that would make scientific sense, it wouldn't necessarily make financial sense ;)

To answer your question. My theory is that my site dropped in search engine results pages for Google. The Bing toolbar stopped getting click info for my site and dropped their rankings as well. That's how the toolbar could gather that info from SERPs alone.

Well, but, so what? Is it the apparent degree to which Bing appears to be weighting information from the toolbar? Otherwise, I don't see how this is new.

The difference is that this was not observed from a sting operation where Google tried very hard to show that their rankings were having an effect on Bing results. It's a happy accident that I decided to take a closer look at and write my thoughts up on. And I'm sure there's more evidence living in Google Analytics that might make an interesting research paper.

Redirecting on user agent to a mobile site is kinda annoying, but not against Google TOS. Only if you show Googlebot different content than you would show other user agents (single out Googlebot), is it considered cloaking.

You talk of a penalty, where a search engine accessibility issue seems a lot more likely. SE accessibility issues usually translate to all search engines.

Besides that there are a few other possibilities that follow Occam's razor and don't promote the Bing-steals idea. The biggest of these would be trends or market changes. If all around the board you get less people searching for your product (for whatever reason), this should reflect in all search engines' stats.

It's also perfectly possible that an older niche player cleaned up his/her game, or a new competitor hit the scene. Both could eat up your traffic.

Or a webmaster that had many sites linking to you forgot to renew his/her domains.

I don't fault the OP for jumping to conclusions (well, maybe a little bit more care would be appreciated); search engines are a black box, and it is all too easy to connect changes you made on the site, to changes in stats. Even when they are not correlated.

Next time when you encounter a bump and ask yourself: Did I get a penalty? Really try to pinpoint the problem: Which parts of the site took a drop? Does that site still link to me? Is is a trend, and do last year's stats show the same decline? etc. etc.

http://www.seomoz.org/blog/whiteboard-friday-oh-i-got-a-pena... can help you with this. Also read up on how to submit a mobile sitemap and have these sites running next to each other without any (perceived) problems: http://www.google.com/support/webmasters/bin/answer.py?answe...

Thanks for your comment! I actually forgot to show the same mobile content to Googlebot-mobile, so I think that's why they assumed it was cloaking.

My site gets pretty consistent traffic. It goes up and down depending on the day of the week and the season but otherwise it's very normal. I am fairly certain that the drop (and now the recovery) has to do with the mobile user agent change I made. The results are now recovering to pre-dip levels, so I don't think it could be something else.

The comments on this post are really illuminating. Thanks for sharing. This is why I really love HN.

Redirecting on user agent to a mobile site is kinda annoying

Indeed annoying, a few weeks ago with the release of FF4 there were sites that mis-classified the browser as mobile, and redirected you to the mobile site. Usually the mobile site is limited and offers no way to switch to the real thing.

Reply on news.ycombinator.com