Hacker News Reader

Comment by fsckboy | Hacker News Reader

nobody "scrapes" soup kitchen soup because it's impractical/expensive/not worth it to do that. Not so with webcrawlers, large numbers of them can, at low cost to the scrapers, use all of your servers' resources, and in Amazon's case, use it to work against your business.

When a person goes to Amazon and looks at what is splashed on the page, there are any number of chances that they will click "Buy!" on any one of them, and winning that lottery is Amazon's business; when a crawler does, it does more "looking" with no chance of purchase, and the data is then used to reduce the value of Amazon's lottery game.

I'm not arguing whether or not crawling is or should be legal, simply saying the "it's not theft because it's copying" argument is inadequate to the task.

View on HN

The fact that you’re trying to use Amazon to convince others that its somehow a negative to use web scrapers is just sad. I’m sure they don’t like it, and that’s in part of why I like them.

Amazon is inherently anti-consumer, and uses every single thing it can to advertise to and/or profit off of you. Their 1984-inspired “security” and home automation systems, their app/website, their policies are all meant to take your money. Which is why web scrapers like CamelCamelCamel are good, because with all of this anti-consumer garbage Amazon shoves at you, you have the power to turn the tables and pick up what you need at the price you want.

The only time I’ve heard scrapers/indexers being a problem is Bing was terrorizing someone’s website, so they just banned every Google/Bing/Yahoo IP. A problem that $1.3T companies don’t have.

You can claim that “its not theft because it’s copying” is inadequate, but I would say the same about your own argument. Because there is no good argument against scraping data. It’s the only way to have a free Internet.

I said I was not making an argument about scraping, just that "copying doesn't deprive anyone else" does not capture the pain point the people building websites complain about.

the rest of what you wrote is a combination of marxism--why can't society cooperate to meet my needs!?--and laissez-faire capitalism--bastards think they can use technology against me, I'll use it against them--neither of which either is a good way to run an economy.

How the Hell did you get two wildly different economic beliefs from what I wrote? And is that meant to be some sort of “gotcha”?

I got it the same place you got your long discursive answer to my only point, where I was simply saying "the copying is not theft because you still have your copy" argument is not adequate to explain the disagreement over scraping.

don't worry, we don't "gotcha" here, we got you, brother!

These are two different things though: vandalism for example isn't theft, yet no one argues it should be legal. This is just like the piracy debate all over again: It not being theft doesn't mean it is legal or ethical. These are different questions: it's possible to believe it is theft, but moral, or that it is not theft, but is immoral, and still be internally consistent.

Reply on news.ycombinator.com