Hacker News Reader

Comment by codedokode | Hacker News Reader

I think such laws are wrong. Scraping is what Google and Web Archive does and it serves good purposes. For example, one can make an application that compares prices for the same item at different internet shops and helps to find the cheapest offer.

I don't understand what's wrong with downloading the information that is published on a public web server. That is what that server was made for in the first place.

Of course there are people who scrape the website, add advertisement and optimize it to rank better than the original website. But this problem can solved with other measures.

And of course those who do the scraping must limit their request rate so the server doesn't get overloaded.

View on HN

I disagree. The person who owns the server should get to decide who has access and under what circumstances. Joe Scraper, having invested nothing, has no claim or rights to it whatsoever.

Furthermore, the fact that the server gives 200 responses is not sufficient implied permission IF a no-scraping policy has been communicated in some other way such as robots.txt or (clearly communicated) TOS.

The techno-nihilist argument of "i'm just throwing bits on the wire, they're choosing to respond" is pretty unconvincing in a world where intent and context matters.

The person who owns a server can restrict access by adding a login form and creating user accounts. Or taking other measures (for example, banning IP networks or countries or implementing a captcha).

I think that the only thing that has to be regulated is the load (number of request per unit of time) on the server. So that it doesn't prevent server from serving pages to other visitors.

Regarding robots.txt, I am not sure if ignoring it should have legal consequences. It is just a hint to robots, don't visit these pages or we will ban you.

If there is no harm done to the server I don't see any problem with scraping.

> The person who owns a server can restrict access by adding a login form and creating user accounts.

Or, say, returning "403 Forbidden" or using a CAPTCHA?

This feels a lot like the "downloading mp3 is stealing".

If you don't want people stealing your music, don't put it other people's hands. The minute you release it to the world, it can't be reversed. See streisand effect.

Likewise, you cannot place burden on your visitors to read and analyze ToS with their lawyers and submit an official request via fax.

If you don't want people access your server, put it behind a paywall or a login screen at least so that you can easily ban people who don't play by your rules.

Otherwise, you have no excuse. If it's publicly accessible, then you cannot enforce any legally binding agreement as you've left the front door wide open and expect people to read tiny letters you put in front of your mailbox.

ToS is a one way street.

Seems like your argument is based on the idea that it's unreasonable to expect the end user to understand the terms under which a service is offered. And that's fair in some cases, but it's also kind of a cop out.

"How was I supposed to know they didn't want me to scrape it? I'm just an innocent passerby dropping bits on a wire" is bullshit in many, many cases. You do know, or at least could easily find out if you wanted to, but choose to maintain a fiction of ignorance to avoid responsibility.

Sure, if someone makes it difficult or arcane to read and understand their TOS, you're probably not (morally) bound by it. But if you close your eyes and plug your ears, you don't have much of a leg to stand on.

Scraping a website can be compared to only downloading mp3 file, not redistributing it to other people.

And I don't see how electronic ToS have any legal power. It is not a contract one has signed before visiting a site. I think ToS should only bind the website owner, not its visitors.

It doesn't. It would be like stepping in to a store because people were welcoming you and realizing that by sitting down you've automatically agreed and consented to tiny ToS written behind every napkin.

It's unenforceable and has never stood up in court.

If someone breaks the TOS, you can block them or send them C&D letters.

If you have the cash to take someone to court but you are going to be on the losing side.

ToS is not a mutually binding agreement. Even if you have a checkbox that says "I have read the terms", it's often thrown out because nobody expects you to read EULA down to the letter.

Is there a law requiring me to read and accept TOS or robots.txt before opening a website? What is the legal status of those? What if TOS says "you owe us $100 for every byte included in this TOS"? What if I have my own TOS about opening their links where they owe me $100 per byte sent?

No. TOS is not mutually binding agreement. You can't invite people to your garage sales and tell people that by looking at the items they've made a final sale automatically without their consent.

Not true (in the U.S.). Just ask Aaron Swartz.

The fact that it does (or, at least, has been interpreted to) criminalize TOS violations is one of the must important flaws in the Computer Fraud and Abuse Act.

https://www.eff.org/deeplinks/2013/01/rebooting-computer-cri...

Aaron Swartz was a weird one. Some zealous IT guy exaggerated it as if national security data were being scraped when in fact most were just shoddy research papers written at 1 AM before the due date.

All people that landed a hand in this man's death should be charged with manslaughter.

Truly a shame and how draconian laws end up killing the innocent in the United States.

That's fine as a discussion starter, but thinking a law is wrong isn't a reason to break it – if that's what you're suggesting.

That argument often turns into "But I probably won't get caught" which is a similarly weak defence.

I am not suggesting to break a law (I don't live in US so that law doesn't apply to me anyway). I just try to point out that it is wrong and serves only publishers' interests.

Reply on news.ycombinator.com