So Cloudflare now wants to collect money to not block people. Is that about the gist of it?
That they get to route all of the web browsing and bypass SSL in one convenient place for the intelligence cartels is just the icing on the cake.
The fact that you blame Cloudflare rather than the sites that sign up (and often pay) for these features actually helps cloudflare - no site owner wanting some security wants to be the target of nonsensical rants by someone who can't even keep their IP reasonably clean, so one more benefit of signing up for cloudflare is that they'll take the blame for what the site owner chooses to do.
Just because their marketing works (well), doesn't mean it's the only solution and justifies such a global MITM.
> nonsensical rants by someone who can't even keep their IP reasonably clean
Says who? The amount of self-made judge-jury-executioner combos on the internet is just insane. Why should we _like_ one more in the mix?
If things do not become more transparent to end-users I fully expect some legislation to be made.
Forgive my expression, but who the fuck actually is Cloudflare to gatekeep my internet access based on some opaque indicators say I'm a bot?
Cloudflare is in no way gatekeeping your internet access. Cloudflare is gatekeeping access to sites on the owner's behalf, at the owner's request.
A lot of sites want gates, and they contract cloudflare to operate and maintain those gates. If it wasn't cloudflare it would be some other company, or done in-house. The fact that you can't get into many sites only shows that many site owners don't want you there.
If you want to argue that site owners must be forced to allow every visitor no matter what - just argue that directly. Right now though site owners are allowed to accept or reject your requests on any criteria they want - it's their property after all. Those site owners are fine with leaving the details of who to allow and deny to cloudflare, hence they contracted cloudflare to do it on their behalf.
> Says who? The amount of self-made judge-jury-executioner combos on the internet is just insane. Why should we _like_ one more in the mix?
Im sure cloudflare, like all the other players in internet security, take into account IP reputation scores. It's a common and fairly effective tool.
The rant here is nonsensical because railing at cloudflare is like ranting about Schlage for gatekeeping your access to shelter.... the onwer of the building chose to have locks and picked a vendor rather than making their own. Much like cloudflare.... Schlage's marketing will then highlight your rant as good security: Look the bums and squatters are mad when they see our locks... do you really want to trust another vendor.
Another reason it's nonsensical is this:
> justifies such a global MITM.
It only does MITM on sites that sign up for cloudflare. It's not global - any site that isn't behind cloudflare is not MITMed. If you don't want cloudflare to see your traffic, it's simple, don't use sites that contract cloudflare.
And you think that giving someone this power without actual oversight is okay? It really isn't.
> ranting about Schlage for gatekeeping your access to shelter.... the onwer of the building chose to have locks and picked a vendor rather than making their own
Except they randomly find some people's "key" incorrect without giving them any recourse.
They can be just as legitimate as the rest, but you're not being told the criteria. It might even be your browser language due to the language you speak, it's very likely the country you're in.
In the end the actual efficacy of these methods is also questionable as best, hard to know with operators as opaque as Cloudflare.
> It only does MITM on sites that sign up for cloudflare. It's not global - any site that isn't behind cloudflare is not MITMed. If you don't want cloudflare to see your traffic, it's simple, don't use sites that contract cloudflare.
Except you don't get a warning before you actually try to enter. It can be added at any point. Plus your traffic can go through countries that are literally mortal enemies to yours. It's not simple and it's dishonest to claim it is.
In the end, sure you might have that freedom to restrict as you wish, but someone shouldn't be doing it at this scale without informing people and without oversight.
Who is overseeing who in your scenario? I think the decision is up to the company doing the contracting. They get to choose how to handle it - if they don't like the results, operations or anything else about Cloudflare they should cancel the contract and get a new vendor. If they are fine with those and want to keep it, they can do that too.
> Except they randomly find some people's "key" incorrect without giving them any recourse.
If my apartment key doesn't work, I don't contact Schlage, I contact the rental company. They may send a new key, or fix the door/lock, and even work with Schlage to fix some root problem. My contact point is still only the company I have a relationship with.
Of course the analogy breaks down here - because in the public web case it's often more like the door to a grocery store. If that is stuck locked and the store can't open, you contact the store - they'll work with their maintenance and vendors to let you in. Until its fixed they just say "sorry you don't get in", and maybe they decide to ban you for making trouble (not good business, but the store gets to do that if they want).
Lets stick with that example and generalize it to all places of business. Plenty of them have security that can ask you to leave and refuse you entry. Bars have bouncers, mall have "cops", office buildings have receptionists and "cops" - in any of those cases they can ask you to leave the premesis, or prevent you from entering the premesis and they don't have to tell you why or give you a course to remedy it. Why do you expect cloudflare to tell you why you can't access a business that doesn't want your traffic?
If you can't get to a site, contact the site owner and ask for them to figure out how to let you in - they may say no, they may tell you that they don't care if they get your traffic, or the may tell you that they'll contact cloudflare and maybe you'll see a resolution.
> Except you don't get a warning before you actually try to enter. It can be added at any point.
Again - a company can refuse your business or your entry, and they don't have to warn you in advance or tell you why. They can even change their rules without warning or explanation. If you have some sort of business with them, and they want to continue it, you have all sorts of recourse - you can call them, get a lawyer to send threatening letters or sue them, or stop paying them since they aren't fulfilling their end of the contract. Your only contract with random public websites is the HTTP protocol - even that has all sorts of "reject without explanation" options - sure they could set up error codes correctly, or just always return 500 or whatever.
> In the end, sure you might have that freedom to restrict as you wish, but someone shouldn't be doing it at this scale without informing people and without oversight.
Someone shouldn't be providing a service that people want for their sites? There can't be a business that helps people who don't want your traffic to actually reject your traffic?
Again who is overseeing who? The site owner is allowed to reject your traffic - either they don't want your traffic or they don't care if they don't get your traffic. The owners have done a cost-benefit analysis and have decided the cost of your traffic does not outweigh the benefit of using Cloudflare to reject it. I don't see how this is Cloudflare's fault.
It seems to me that you've been deemed as "not worth the hassle" and that sucks for you. I just don't see that makes Cloudflare the bad guy - if you actually are worth the hassle, talk to the people responsible for the site about why you are worth the hassle and get them to make the situation right, they are the ones who hired cloudflare and decided you weren't worth the hassle to begin with. They are the ones who can change their setting or their vendor or whatever, not the company that was hired to execute a contract on the site owner's behalf.
The sum of all websites contracting CF can be more damaging to me as an individual than it is to the companies doing the contracting. So I should definitely have a say on how they operate.
> If my apartment key doesn't work, I don't contact Schlage, I contact the rental company.
Yeah, except with your analogy it's Schlage that's breaking or fixing your keys, not the rental company. You don't know why or now. Some days it takes more effort than others to open your door. Your option is of course to move apartments, but you can't boycott Schlage, if the landlord decides to use them.
> Why do you expect cloudflare to tell you why you can't access a business that doesn't want your traffic?
Because for example if it's based on my native language or religious preferences it would be literally illegal in real life.
> Someone shouldn't be providing a service that people want for their sites? There can't be a business that helps people who don't want your traffic to actually reject your traffic?
Yeah, they should not be able to do so without someone overseeing that they aren't blocking accessibility-oriented browsers or discriminating based on non-technical factors that just happen to correlate in some other way.
> It seems to me that you've been deemed as "not worth the hassle" and that sucks for you.
I hope you get to enjoy kafkaesque technical obstacles thrown at you for no fault of your own, and I hope that sucks for you.
1. yes they do. have you ever been to vegas? there's cameras and facial recognition everywhere. outside of vegas, some bars and clubs also use ID scanning systems to enforce blacklists, and in most cases that system is outsourced to an external vendor. finally, ticketmaster requires an account to use, and to create an account you need to provide them your billing information. that's arguably more intrusive than whatever cloudflare is doing, which is at least pseudonymous.
2. "profiling people" might be objectionable for other reasons, but it's not a relevant factor in whether something is a "protection" racket or not. There's plenty of reasons to hate cloudflare, but it's laughable to describe them as a criminal enterprise.
2. Of course it is relevant. Because the more false positives they have the more money they can extort. They have negative incentive for their system to work properly.
P.S. ticketmaster is absolutely criminal, too.
What are the "false positives" in this context? It's specifically for blocking bots, and enrollment into the program to get unblocked is designed for bot owners. It's obviously not designed to extract money from regular users. I doubt there's even a straightforward way for regular users to pay to get unblocked via this channel. As the people who are running blocks and are blocked, I don't see what the issue is. Isn't it working as intended by definition?
Define "bots" in a way computers can understand.
> What are the "false positives" in this context?
Regular users that cloudflare (profiles) accuses of being bots. God help you if you want to block trackers or something else that's not regular.
> I doubt there's even a straightforward way for regular users to pay to get unblocked via this channel
This is part of the problem. But hey, at least they are only a process change away from charging normies too.
How is having a specific definition relevant to this conversation? An approximate definition of "a human using a browser to visit a site" probably suffices, without having to get into weird edge cases like "but what if they programmed lynx to visit your site at 3am when they're asleep?".
>Regular users that cloudflare (profiles) accuses of being bots. God help you if you want to block trackers or something else that's not regular.
I use ublock, resistfingerpnting, and a VPN. That probably puts me in the 95+ percentile in terms of suspiciousness. Yet the most hassle I get from cloudflare is the turnstile challenges can be solved by clicking a checkbox. Suggesting that this sort of a hurdle constitutes some sort of "criminal enterprise" is laughable.
I do occasionally get outright blocked, but I suspect that's due to the site operator blocking VPN/datacenter ASNs rather than something on cloudflare's part.
>This is part of the problem. But hey, at least they are only a process change away from charging normies too.
So they're damned if they do, damned if they do? God forbid that site operators have agency over what visitors they allow on their sites!
Because it's a computer that automatically does it. That's the entire problem here. Humans are not in the loop, except collecting the paychecks.
> An approximate definition of "a human using a browser to visit a site" probably suffices
Humans are not doing the blocking. "Approximate" is not good enough when, for example, I need to go to a coffee shop and use an entirely different computer to trick cloudflare into letting me order from my longtime vendor. And I must repeat that my work computer is doing absolutely nothing interesting. My job and livelihood depend on this.
> without having to get into weird edge cases like "but what if they programmed lynx to visit your site at 3am when they're asleep?".
What about an edge case like 'using your bone stock phone to visit a site once'?
What about all the poor suckers that installed an app that loaded legal software designed specifically to use their phone's connection for scraping a la brightdata? Residential proxies are big business.
There are billions of users on the web. It is one gigantic pile of edge cases. And that's entirely the point. CF may get some right but they also get plenty wrong with no recourse (but now you may be allowed to pay them money for access).
> So they're damned if they do, damned if they do?
Yes. Their entire business model is "we have a magic crystal ball that only stops 'the wrong people'™ from your website".
> God forbid that site operators have agency over what visitors they allow on their sites!
They quite literally don't have that agency. This goes back to "define bot". There are zero websites that would want to block me from making purchases from them and yet that is exactly the result in the end. I had to change vendors for a five figure order because I was up against a deadline and couldn't get around the cloudflare block from my office, and the vendor had closed for the night so I couldn't call them and bypass the whole mess.
Afterwards we spent nearly a week trying to figure out how to let me buy from them again and they were willing to keep going back and forth with CF on my behalf but I was over it and not going to spend any more time. Now I'm using the non-CF vendor to their disappointment. So much for agency.
> I use ublock, resistfingerpnting, and a VPN. That probably puts me in the 95+ percentile in terms of suspiciousness. Yet the most hassle I get from cloudflare is the turnstile challenges can be solved by clicking a checkbox.
Good for you? I have a bone-stock computer on its own connection just to try to work around this BS and yet I still sometimes get an infinite loop where the checkbox never goes away.
When I have my VPN to our euro office on I am 100% unable to access CF sites whatsoever. Been that way for as long as I can remember.
I don't see how "Humans are not in the loop" is a relevant factor for whether something is a "criminal enterprise" or not. Humans are often not in the loop in approving loans/credit cards either. That doesn't make equifax a "criminal enterprise" for blocking you from getting a loan because you can't pass a credit check. Even in jurisdictions with laws against automated decision making by computers, you can only seek human redress in specific circumstances (eg. when applying for credit), not for whether a website blocked you for being a suspected bot or not
>I need to go to a coffee shop and use an entirely different computer to trick cloudflare into letting me order parts on digikey. And I must repeat that my work computer is doing absolutely nothing interesting. My job and livelihood depend on this.
1. At least looking at the response headers, digikey.com is served by akamai, not cloudflare
2. I can visit the site just fine on commercial VPN providers. Maybe there's something extra sus about your connection/browser, but I find it hard to believe that you have to resort to getting a separate computer and making a 10 minute trek to visit a site
3. like it or not, neither cloudflare nor digikey has any obligation to serve you. They can deny you service for any reason they want, except for a very small list of exceptions (eg. race or disability). "browser/configuration looks weird" is an entirely valid reason, and them denying you service on that basis doesn't mean cloudflare is running a "protection racket".
>What about an edge case like 'using your bone stock phone to visit a site once'?
that's clearly not an edge case
>What about all the poor suckers that installed an app that loaded legal software designed specifically to use their phone's connection for scraping a la brightdata? Residential proxies are big business.
That's a false negative, not a false positive. Maybe the site operator has a right of action against cloudflare for not doing their job against such actors, but you have no standing when you're blocked and they're not.
>Yes. Their entire business model is "we have a magic crystal ball that only stops 'the wrong people'™ from your website".
And do they actually claim 100% accuracy?
>They quite literally don't have that agency.
They can go with another anti-bot vendor. Competitors such as imperva or ddos-guard use similar techniques because it's the state of the art when it comes to bot detection.
>This goes back to "define bot". There are zero websites that would want to block me from making purchases from them and yet that is exactly the result in the end. I had to change vendors for a five figure order because I was up against a deadline and couldn't get around the cloudflare block from my office, and the vendor had closed for the night so I couldn't call them and bypass the whole mess.
>Afterwards we spent nearly a week trying to figure out how to let me buy from them again and they were willing to keep going back and forth with CF on my behalf but I was over it and not going to spend any more time. Now I'm using the non-CF vendor to their disappointment. So much for agency.
I'm sorry this happened to you, but any anti-fraud/bot system is going to have false negatives and false positives. For every privacy conscious person that's making a legitimate purchase using TOR browser and delivering to a different shipping address, there's 10 other fraudsters with the same profile trying to scam the site. This is an extreme example, but neither the business or cloudflare has any obligation to serve you.
>Good for you? I have a bone-stock computer on its own connection just to try to work around this BS and yet I still sometimes get an infinite loop where the checkbox never goes away.
What OS/browser (and versions of both) are you using?
>When I have my VPN to our euro office on I am 100% unable to access CF sites whatsoever. Been that way for as long as I can remember.
sounds like their residential proxy detection (that you were asking about earlier) is working as intended then :^)
I edited them out because they were only one of many problem sites.
> Maybe there's something extra sus about your connection/browser, but I find it hard to believe that you have to resort to getting a separate computer and making a 10 minute trek to visit a site
Maybe half a decade ago someone had malware from my IP. Maybe my router's mac address was used by some botnet software somewhere. Maybe I'm on the same subnet as some other assholes.
> 3. like it or not, neither cloudflare nor digikey has any obligation to serve you. They can deny you service for any reason they want
The vendor in question (this one was not digikey) very explicitly wanted me as a customer.
> them denying you service on that basis doesn't mean cloudflare is running a "protection racket".
Them charging to correct their mistake is.
> that's clearly not an edge case
That's my point. I know for sure that vanilla android on t-mobile periodically gets the infinite loop in this area of my city. It usually goes away within a week but there's no rhyme or reason.
> What OS/browser (and versions of both) are you using?
I have seen it on linux windows and android.
> sounds like their residential proxy detection (that you were asking about earlier) is working as intended then :^)
I don't understand this. They have a normal ISP in a business district?
ETA: I have less issues on my home computer, which browser extension'd up, ironically enough.
But the fact that other security providers flagged your IP/browser should be enough to conclude that cloudflare isn't engaged in some sort of "protection racket" to extract money from you?
>The vendor in question (this one was not digikey) very explicitly wanted me as a customer.
Most e-commerce vendors also want customers as well, the problem they can't tell an anonymous visitor a legitimate customer or not, so they employ security services like cloudflare to do that for them.
>Them charging to correct their mistake is.
It's unclear whether the cloudflare product actually constitutes "Them charging to correct their mistake". For one, it's unclear whether you're blocked by cloudflare or the site owner, who can also set rules for blocking/challenging users. Moreover, it's unknown whether the website owner would opt into this marketplace. Presumably they're blocking bots for fraud/anti-competition reasons. If that's the case I doubt they're going to put their sites up for scraping to make a few bucks. Finally, businesses are under no obligation to give you free appeals, so the inability for you to freely appeal doesn't constitute a "protection racket".
>vanilla android on t-mobile periodically gets the infinite loop
>I have seen it on linux windows and android.
you must have a really dodgy IP block then.
>I don't understand this. They have a normal ISP in a business district?
Its probably generating two signals associated with fraud:
1. high latency means than a proxy is being used. This is suspicious because customers typically don't VPN themselves halfway across the world, but cybercriminals trying to cover their tracks by using residential proxies do
2. "business" ISPs might get binned as "hosting" providers, which is also suspicious for similar reasons (eg. could be someone using a VPS as a proxy).
Sure, the unlucky few who accidentally does some online shopping when connected to their work VPN might get falsely flagged, but they probably figure it's a rare enough case that it's worth the loss compared to the overwhelming amount of fraudsters that fit the same pattern.
Most websites aren't "open to the public". Most use firewalls, configure rules, etc that already block certain accesses. It's open to selected groups, just maybe including 1s you're allowed to be a part of.
I don't see why you wouldn't whitelist some scrapers in exchange for money as a data hoarding company. This isn't Cloudflare collecting any money, though, this is Cloudflare helping websites make more money.