Ask HN: How to Make Money from Web Scraping? | Hacker News Reader

Ask HN: How to Make Money from Web Scraping?

17 pointspythonbase12 comments

What are some ways to make money from web scraping?

Few options that I can think of, are:

1. Scrape and sell data

2. Scrape public data and publish as a website. Make money with ads / subscriptions

3. Build price comparison websites

4. Offer web scraping services.

Any real life examples / use-cases out there?

View on HN

If any HN community members are doing web scraping that may benefit from residential proxies rather than datacenter IPs, I can get you set up with some free test credits on our residential proxy network: https://packetstream.io

We tunnel requests through real opt-in residential ISP users.

Send us a message via our contact form and we'll load up your account with some test credits.

Happy scraping!

Sounds a lot like you're resnet:

https://krebsonsecurity.com/2019/08/the-rise-of-bulletproof-...

That's badass but how do they stop from being banned?

Like if I scrape Craigslist enough and get that IP banned the resident can't visit Craigslist?

I don't know enough about this to understand.

Or would it cycle through the proxy IPs real fast?

> Or would it cycle through the proxy IPs real fast?

Exactly. PacketStream lets you change your visible IP with each request. If you need a persistent IP/geolocation that can be configured as part of the request.

That is totally badass. Thanks.

usually these are controlled via an api of some sort. depending on the distributed system. another popular solution is hola which offers basically the same thing, albeit I think worse prices when I was using it than this one...

The coolest use of a botnet I’ve seen in a while!

Possibly one more item is "scraping to gather analytics and trends", the revenue comes from using machine learning to predict trends based on the scraped data from social media, news etc... in most cases the raw data isn't as valuable as the patterns or predictions you can see in it...

Example -> https://meetglimpse.com/

Very interesting. Your project?

Not my project, was on "Show HN" a few months ago.

But I did try to use Web-Scraping a few years ago, when I was trying to set up a crypto-currency [1] arbitrage... but there were a lot of risks and challenges... could be another option for you though.

[1] https://en.wikipedia.org/wiki/Arbitrage

So not to be decisive... but if you have a skill such as a talent for scraping why not look for a value-add for the world compared to primarily "rent-seeking" activities like the above. (Advertising is never a good way to make money)

Why not set up a scraping system that helps people improve accessibility by alerting them to flaws in their sites?

Good idea. Can you elaborate this a bit?

Reply on news.ycombinator.com