Hacker News Reader

Comment by VladVladikoff | Hacker News Reader

This is actually a commonly known fact. There are many services now that sell “residential proxies”, which are always mobile IP addresses. Since mobile IPs use CGNat it’s also not great to block the IP because it can be like geofencing an entire city or town. Some examples are: oxylabs, iproyal, brightdata, etc.

Recently I filed an abuse complaint directly with brightdata because I was getting hit with 1000s of requests from their bots. The funny part is the didn’t even stop, after acknowledging the complaint.

View on HN

The "compliance officer" at Bright Data, instead, offered me a special deal to protect my site from their bots ... they run a protection racket along with all the rest of their nastiness.

I worked for an Amazon scraping business and they used Luminati (Now Brightdata) for a few months until I figured out a way to avoid the ban hammer and got rid of their proxy.

They indeed provided "high quality" residential and cellular ips and "normal quality" data center ips. You had to keep cycling the ip pool every 2-3 days which cost extra. It felt super shady. It isn't their bots, they lease connections to whoever is paying, and they don't care what people do in there.

> ... until I figured out a way to avoid the ban hammer ...

You had my curiosity ... but now you have my attention.

Without bothering to check on Amazon, I successfully scraped meta stuff for years at rates exceeding 20gbit/s without any proxies but just rotating IPv6 addresses on the same couple of blocks for every request

There are usually silly bypasses like this that easily work even with bigco stuff

They provide an SDK for mobile developers. Here is a video of how it works. [0]

[0] https://www.youtube.com/watch?v=1a9HLrwvUO4&t=15s

WOW that video! Ain’t no way anyone has EVER read those terms. This feels so insidious that it really should be illegal. Wonder if this exists in the EU or if they have shut it down already?

That video has the app asking the user to confirm the use of their device to run a proxy within the app - but is there any hard requirement for this, could apps use this SDK and silently run as a proxy?

My take is it's mostly irrelevant, but read the lobsters post mentioned elsewhere.

Yes, and it doesn't matter if they do read the terms- to the average user they sound totally innocuous, especially placed next to a big shiny "GET 500 FREE COINS" button.

I suspect most people, even when told exactly what the app using that SDK would be doing, wouldn't actually see the potential problems...

Until one day, they get swatted for accessing child porn.

Actually, that might be one way to draw attention to the problem. Sign up to some of these shady "residential proxy" services, and access all sorts of nasty stuff through their IPs until your favorite three-letter agency takes notice.

Also see https://www.youtube.com/watch?v=AGaiVApKfmc - "Avoid restrictions and blocks using the fastest and most stable proxy network"...they're pretty upfront with this, aren't they?

Oh, and they will sell you the datasets they've already scraped using mobile devices: https://brightdata.com/lp/web-data/datasets

This actually explains a phishing attack where I received a text from somebody purporting to be a co-worker asking for an Apple gift card. The name was indeed an employee from a different part of the large company I worked for at the time, but LinkedIn was the only possible link I could figure out that was at least somewhat publicly available information.

This should probably be required in all CS curriculum: https://ocw.mit.edu/courses/res-tll-008-social-and-ethical-r...

It should be illegal, but this stuff is propping up the appearance of a healthy economy so nobody will touch it.

That scam definitely uses linked in as the source. We get a lot of those BEC emails and it’s always the people who are on LinkedIn. Also keep in mind LinkedIn has had big database leaks in the past, you might not even need to scrape them, just download a huge database from a leaks site.

IMO Google Play should check apps for presence of this SDK and other similar SDKs, and, upon detection, treat these apps as malware.

I was wondering if they already do but maybe it’s a cat and mouse game where those companies obfuscate their code to avoid automated detection.

That's sleazy. It's slipping drugs into a kids lunchbox and letting smuggle it across the border..

Lately Reddit has been showing me posts in subreddits for some of these services. They pitch "passive income" by sharing your connection, an easy way to make a few bucks by renting out your unused capacity. What happens is that you become an endpoint for their shady VPNs. These subreddits are full of people complaining that they're getting hit by abuse complaints from their ISPs. Naturally, these services claim to forbid any nefarious activity, and naturally they don't actually care.

Salad, right? What a strange business

Why is it strange. Of course it exists.

One such example is brightdata, on lobsters someone did a writeup

https://lobste.rs/s/pmfuza/bro_ban_me_at_ip_level_if_you_don...

Never heard of lobsters before. Cool site. Seems to be invite only though :( If you could share an invite that would be cool. torosanchez@protonmail.me Thanks!

Reply on news.ycombinator.com