Hacker News Reader

Comment by dexen | Hacker News Reader

The newspapers went to court asking for broad relief (unlisting from google results), instead of putting in place proper META tags and /robots.txt to direct what may be, and what may not be indexed and cached. The court handed them just as much of the proverbial rope as they asked for.

Google's web crawlers asked the WWW servers to GET, were served with HTTP 200 OK, found no prohibition in /robots.txt and so followed the standard protocol.

Note Google Web Search and Google News use separate bots (Googlebot and Googlebot-News) to give website owner full control.

View on HN

I can't decide whether I find things like this hilarious or disturbing. They could have fixed it with engineering, but they were either so uninformed or incompetent that they tried to fix it with a lawsuit.

Or maybe one incompetent overrode them all.

Manager: Evans, get in here! Why are our news articles all over The Google?

Evans: Um, sir, they index our -

Manager: I don't want to hear your technical mumbo jumbo! Get legal on this to get it fixed YESTERDAY!

Evans: There's no need, it's literally a 5-minute fix, I could just add an entry to robots.txt while the coffee is brewing...

Manager: What did I just say about mumbo jumbo? Get out of here, and send in Stevens from Legal on your way.

Again, the lawsuit was over Google News' display of lead paragraphs and other excerpts from articles. It didn't pertain to the general search engine.

The lawsuit may not have, but the ruling certainly and explicitly does.

That's incorrect. The court's ruling specifically pertains to the use of article content in the Google News aggregator and its storage in Google's cache. There is no mention of simply linking to articles in the search engine, nor would there be reason for one since the lawsuit was over the use of copyrighted article excerpts, not hyperlinks.

What Google is doing is akin to the punishments Microsoft used to dole out in the 90s, such as increasing Windows OEM licensing fees or threatening to revoke licenses entirely. Google is playing a dangerous game in monopoly-conscious Europe.

Google uses separate crawlers for its search engine and its news service. Blocking the news service really would have been a trivial edit to their robots.txt file.

The lawsuit was over the use of lead paragraphs in the Google News aggregator, not the general search engine. Google is intentionally misinterpreting the verdict of the lawsuit in order to punish those who might challenge the company. Remember that Google also defied the court order to publish the result of the lawsuit on its Belgian sites, so it's obviously not afraid to thumb its nose at the courts.

"Nice online newspaper you've got there. Hate to see something happen to it."

> Google is intentionally misinterpreting the verdict of the lawsuit in order to punish those who might challenge the company.

Even if we stipulate this, how is that bad? Should Google be forced to deal with companies that screw with it? How far do you want to take that idea?

You're asking me how it's bad that Google is using a strong-arm tactic to punish content-owners who sued it? Surely, I don't need to answer that question, especially with Google's monopoly troubles in Europe. Google should let things be, respect the decision, and move on.

I'd also like to mention again how incredibly lame it is that my comments are being voted down in this discussion. All I did was correct people who assumed the lawsuit was pertaining to ALL search engine links. I'm at a loss as to what I wrote that warrants negative votes.

Didn't you notice every second link here is about Google? It's not permitted to say anything bad about Google here.

> You're asking me how it's bad that Google is using a strong-arm tactic to punish content-owners who sued it?

Yes, I am. If you sue someone, can you really expect them to be nice to you afterwords?

The newspapers went to court asking for broad relief (unlisting from google results)

They went to court over Google News copying excerpts from their articles, such as the lead paragraphs, and displaying them in the Google News aggregator. The lawsuit didn't pertain to the general search engine.

That might have been the intent of the newspapers, but that wasn't the court order their lawyers got them.

That's incorrect. You can read the ruling for yourself:

http://www.copiepresse.be/pdf/Copiepresse%20-%20ruling%20app...

The decision was pertaining to the use of lead paragraphs and other excerpts from copyrighted articles in the Google News aggregator. It didn't refer to plain links in the general search engine.

I see that my comments in this discussion are being voted down for pointing this out, which is bizarre. I'm simply stating the truth and correcting posts that mistakenly assume the lawsuit was in reference to ALL links in the search engine. Voting me down is incredibly lame.

I believe that Google is deliberately misinterpreting the decision, likely to remind content-owners that Google controls their source of online revenue and to therefore back off. Google has a history of being defiant with regards to this lawsuit, ignoring a court order to publish the court's decision on their Belgian websites, so they would surely have no problem doing something retaliatory under the guise of following the court's decision.

OK, so I read the PDF you linked to. Page 48, under the heading "V. ENACTING TERMS", point 3:

--- Begin Quote --- Reforms the challenged judgment to the extent that it ordered "Google to withdraw from all its sites (...) all the articles, photographs and graphic representations (...)";

Ruling again on this point only, reforms the injunction as follows: Orders Google to remove from the Google.be and Google.com sites, more specifically from the "cached" links on "Google Web" and from the "Google News" service

--- End Quote --- (There may be some errors in the copy above and please excuse my pathetic formatting).

See the thing is you are arguing about what the lawsuit was about, which is different to what the judgment was.

It says there in black and white "to remove it from all sites".

Or am I just reading that wrong?

See the thing is you are arguing about what the lawsuit was about, which is different to what the judgment was.
It says there in black and white "to remove it from all sites".

Or am I just reading that wrong?

You're reading it wrong. It was a copyright infringement lawsuit over the scraped content displayed by Google News and in Google's cache, so the court's ruling was to remove those infringing materials from those services, not to remove the hyperlinks to the original articles in the general search engine.

> I see that my comments in this discussion are being voted down for pointing this out, which is bizarre.

Before posting the above link, You failed to provide quotation supporting your statements, and there were quotes saying otherwise :-)

Let's see, the original ruling went:

  Orders GOOGLE to withdraw from all its sites (notably Google news
  and "cache" google or any other name) all the articles (...)

(page 9, 2nd paragraph)

However, there is amended ruling listed on page 47 reads:

  Orders Google to remove from the Google.be and Google.com sites,
  more specifically from the "cached" links on "Google Web"
  and from the "Google News" service, all the articles (...)

which confirms your story. Thanks for being persistent!

----

At any rate, this lawsuit is bullshit, and should be called out till they blush. The webserver has been likened to house left with doors wide open and robbed in absence of the owner (page 36.). Seriously?

What The Copiepresse actually did was like having welcome guests for a long time, and then suddenly suing them for enjoying themselves.

The argument of META tags and robots.txt was raised:

  In 2006, when the action was filed, the publishers
  had not activated these tags, which allowed Google
  to inventory all their pages, notably "cached".

but rejected on weird reasoning:

  However the parties do not explain how it would be
  technically possible to, via these tags, prevent Google
  from reproducing ((the content)) in the "Google News" service.

(page 35.) I say, it's the same as with law or court orders: doesn't technically prevent, just informs what is allowed and what is not.

----

For a wider picture: the newspapers joined (by publishing public websites) a well established space: the Internet. It takes serious incompetence -- or malice -- to move somewhere and not follow well-documented, mutually-benefiting local rules.

Let's be clear: Google is no schoolyard bully here -- even if it was elsewhere the other day -- and newspapers are no victim. If anything, it's newspapers that advanced from large, slowly grinding company to a fast, focused start-up and found: their playbook hopelessly inadequate, their narrow field of view constraining.

You failed to provide quotation supporting your statements, and there were quotes saying otherwise :-)

The lawsuit is public knowledge, with plenty of articles written and a decision published online. I think we both know that I was getting voted down because I was correcting misinformation about a court decision against Google. :)

If you started a newspaper that did nothing but copied the headlines and lead paragraphs from, say, the New York Times, how long do you think it would be before you got a cease-and-desist letter in the mail? That's copyrighted material.

There's a rule stating that content-owners are supposed to let another company take their work and make money off of it? What rules are you referring to specifically?

Let's be clear: Google is no schoolyard bully here -- even if it was elsewhere the other day

That's arguable, but more importantly, it's a dangerous maneuver for Google in anti-monopoly Europe.

> I was getting voted down because I was correcting misinformation about a court decision against Google. :)

Can't speak for others, but I've voted you down exactly because you were making strong claims -- opposite to sources available to me -- without support (till you linked to that document). One of my mistakes, eh.

To the wit: I don't particularly like Google. But I strongly believe each case should be decided on its own merit, not on popularity (or otherwise) of defendant.

> If you started a newspaper that did nothing but copied the headlines and lead paragraphs from, say, the New York Times, how long do you think it would be before you got a cease-and-desist letter in the mail?

If I started a website that did way more than just copying titles and lead paragraphs from wide range of media sources, how long do you think I'd have to explain to various folks it actually is legal? It's the videocassette recorder lawsuit history all over again [1]. Significant non-infringing uses.

> There's a rule stating that content-owners are supposed to let another company take their work and make money off of it? What rules are you referring to specifically?

You have twisted it strangely @_@ Nobody asks the owners to let others use their content, only to indicate their preferrence.

There is a rule that what is served with HTTP 200 OK (and has such and such META tags and permissive /robots.txt) to a robot is allowed for that robot. Most websites need their content to be included in search results, news or not. Thus the reasonable default.

There is expectation of due dilligence. If you by mistake leave your home open once and somebody robs it, it's a robbery, plain and simple.

But! If you invite somebody to live with you for a good stretch of time, with mutual benefit, you can't then claim they profited against your will. It's your conscious decision, plain and simple again.

The newspapers did the later.

Web crawlers are fact of life, pre-dating the newspapers' webservers. There's an agreed-upon protocol for dealing with them. You either show due dilligence (which, nb., is easy and cheap -- the document you linked states clearly the newspapers put proper META tags in place at some later point) or you admit mistake and clean up your mess yourself.

Want a car analogy? Nobody asks drivers to turn only left. But they must indicate turns unambiguously with signals. The newspapers lit the wrong signal and now want their scratched car reimbursed.

----

At any rate, thanks for bringing the actual ruling to attention :-)

----

[1] http://en.wikipedia.org/wiki/Videocassette_recorder#Legal_ch...

Looking at the text of the ruling, I can see how someone might interpret it either way, either as having to eliminate the links they have "cached" or the links to a "cached" version of the website. I think that if I was a legal counsel to Google I would probably advise them not to do any linking at all in the face of that ambiguity and the large penalties for violating the court order.

Equally important, I'm having a hard time imagining how it would be reasonably possible for Google to only link to the articles. A hyperlink is a combination of both text and the link, and the text has to come from somewhere. Usually it is the title of the web page that is linked to, but then Google would still be stealing content from the newspapers. I suppose they could just use the URL itself as the text of the hyperlink, but then we get to the second part of this issue. Complying with the order and still providing links would require rewriting a lot of Google's front end code just for this one specific case. Given that Google already has a process to block sites entirely at the back end it would be a lot easier to just comply with the court order that way, and probably the only way that they could do it quickly enough.

Reply on news.ycombinator.com