This also puts some of the troubling reports that politicians were bringing phones into SCIFs into context…
On a server that used to get below 1k HTTP requests a day, I've had several cases of mass scraping from these botnets exceeding 100req/s. The web is getting harder and harder to use due to all the anti-bot mechanisms site operators had to put up.
How is DDOSing every site on the web benign?
A compromised device inside the White House network could presumably be doing something _much_ worse than participating in a DDOS against random websites.
> One of the proxy services is a Russian company whose founder’s web hosting business was ground zero for cyberattacks against election systems in Arizona and Illinois in 2016.
Surely a Russian company known for cyberattacks against the US having direct access to the White House network is far more serious than just DDOSing random websites. This is espionage at the highest level. (Again.)
A whitehouse network. It isn't like everything done in that building run through a single wifi router. It is likely an unprotected network for personal use with zero connecting to anything important.
Great, they have some isolated network.
What are the chances it _never_ touches a device with a path to an important computer (e.g., one core router or switch at some point)? Good thing there haven't been any Cisco/Juniper/etc vulnerabilities ever.
What are the chances no important device _ever_ touches this isolated network? No important person ever gets annoyed at the security/filtering on the secured network and decides to connect to the public one instead? The secured one has some sort of issue, so their phone decides to connect to the guest network?
Are they going to compromise the military C&C network in the underground bunker? Probably not.
But it's probably more realistically damaging if they were to compromise some high level staffer's phone and using it as a listening device in a bunch of mid-level Whitehouse meetings.
So you're looking at not just national security stuff, but the ability to get personal data like selfies of fucking goats or screenshots of Maltese & Azerbaijan bank transfers, etc. Makes it easy for nefarious parties to manipulate standing with POTUS and blackmail people. The only escape for many of these people is pardons.
Indeed. I recently set up a bot on my local machine that I can interact with via Telegram. I gave it the ability to retrieve the contents of a URL via Playwright (with modifications). Basic things like getting the Readability view and getting the site as Markdown with links.
I was essentially mimicking the feature OpenAI etc offer where it can do a web search for you and get you the answer you seek. I just didn't want to pay a company for that feature.
It's not autonomous. It only acts when I tell it to. The actual search is via the Kagi API, but the URL contents is via Playwright. I also ensured it can't do lots of requests at a time. There's a mandatory (random) sleep of between 1 and 5 seconds between two requests.
In less than a week, I've been flagged. When I do a Google search (as a human), I have to prove I'm human. Many sites have simply blocked me - I can't buy stuff from various vendors[1], I cannot access several news sites, etc.
The thing is, I didn't really access that many sites with the bot! At most it's the equivalent of someone mildly addicted to the web. Yet they caught me quickly.
Obviously, I disabled the bot's capability to get site content. I wonder how long my IP will be in the blacklist.
I know some people will say "Good riddance" to the outcome and to what I did. I'd love to have a real, serious discussion on whether what I did was immoral. But whatever your thoughts on the matter, do ponder over the fact that the system has made it hard for average Joes like you and me to have this "feature", but if you pay a giant tech company a monthly fee, you can have it.
[1] Thankfully, Amazon didn't block me. In the past I'd avoid them, but now I'm in the dilemma where I don't want to buy stuff from sites that make it difficult for me to buy stuff from them!
I will instead say "skill issue".
Using Playwright results in some very obvious and easy to detect fingerprints. I'd expect better of someone on hacker news.
Mine hasn't been detected as far as I can tell, so I know it's possible to do better.
I normally use a VPN to access the Internet. It's an ordinary consumer-grade VPN, and I use it for pretty innocuous things like watching the international versions of Netflix or avoiding being geographically-profiled and charged more when I book travel. The VPN itself, I'm sure, is being used for all sorts of naughty stuff: when I look it up in whatismyip, it comes up as VPN, "abuse detected", and "residential proxy detected". I expect to get blocked occasionally on it and just pause the VPN then.
What I found surprising is that pausing the VPN doesn't always clear the CAPTCHAs. The next request will also fail, sometimes. For a while I was wondering if I have a residential proxy on my network, but I went hunting with lsof and bandwidth monitoring and a list of all the open connections through my router, and it seems clear. Yet even with VPN disabled, I'll get CAPTCHAs from Google and Cloudflare.
I suspect they're using browser/TLS fingerprinting and then smearing the results of that across bad IPs. If you connect to them through a known-bad IP, they not only will CAPTCHA you, but they will record a fingerprint of any data they can. Then if you connect again with that fingerprint, regardless of the IP, they block that too.
Ironically, if you are actually malicious, you can get around this pretty easily. Because you'll know not to use a fingerprintable browser, or to rotate fingerprints. So you have one completely separate identity on a given IP (that is not yours), and if it gets blocked, you roll up another completely separate identity on a different IP. But this is too much work for an ordinary consumer who just wants to keep their IP private. So you end up with this excluded middle where the only rational strategies are to either do exactly what the website expects you to do (with no funny business, even if it's innocuous), or to go whole-hog as the aggressor where you're determined to skirt their policies no matter what, and if you're doing that you might as well amortize the effort by extracting everything of value you can from them.
That modified Playwright was based on Firefox, and that's what I use for my regular browsing. So that didn't help.
I can actually launch Chrome and not deal with this - but I'm not going to. In fact, what surprises me is that I can create a new Firefox profile and not get those captchas. I don't know how my using that modified Playwright as a bot got them hooked on to my Firefox profile. I suspect they had my FF profile as a "suspicious" entity even before I made the bot (I use a number of privacy/anti-cookie extensions), and the bot pushed the score over the edge.
I'll probably start fiddling with my profile - disabling extensions, until things seem to work. Thankfully, no one else in the household is impacted.
For now, I got around all this by just paying for the Kagi extract API. It's not as good as my modified Playwright had been, though. I thought of a VPN, but I suspect that won't last long (not a residential proxy, etc), and I don't want an ongoing monthly fee. I like Kagi's pay as you go option.
But as I said, it's a weird world where suddenly Amazon is coming out the "better" site/vendor. It's funny - in the old days I was pretty anti-Walmart, but a few years ago I started preferring them over Amazon (and even several local stores) for a number of reasons (better collection than local stores, not having to deal with fakes for online orders, etc).
I've not been liking the state of the web for years prior to the advent of LLMs, but this is feeling like a final straw where I accept "the web is dead". We need a fundamentally different protocol - the model of "Set up a server anyone can reach, put content out there, and then put up a lot of sophisticated technology so your server isn't hammered" is not tenable. The protocol itself needs to manage the load.
I made my modifications at the network level and auto redirect a configurable set of sites to go to reader mode/paywall bypass by default with a MITM proxy and a custom CA cert I trust
While your server handles that, what does your CPU do with the other 99% of available cycles?
I run https://bunnystomp.com on an Always Free VM from a major cloud provider. It handles 1,930 req/s at 33 ms latency with no tuning/optimization. It's literally just python behind cloudflare.
I have no idea what the site you linked does, but it doesn't look like a cgit instance to me; that's what got hammered on my end, and when hosting it, one very quickly finds out that many git operations are CPU-bound.
What are you doing to make this true?
I'm a screen reader user (which is as "weird and unusual" as it gets when fingerprinting mechanisms are concerned), and I am not really noticing much difference versus 2022 or so. Sure, there's little bits of anubis here or there, and a few sites (which I know for a fact are experiencing targeted DDoS attacks unrelated to AI scraping) having annoying Cloudflare verification screens, but that's about it.
My setup (other than the screen reader) is as vanilla as it gets. Safari, AdGuard for Ad blocking, no VPN except work VPN when necessary, no PiHole, no settings changed. If you're using something more esoteric, maybe try going mainstream and see if that improves your experience.
In comparison I get cloudflare recaptcha on every site that uses it, sometimes multiple times per session and sometimes not even letting me pass at all due to "error".
And if the website uses Google recaptcha then I close the tab because I know it will just end up in endless recaptcha hell.
It’s DRM that I think of as silly larping that doesn’t really bother me, but it’s clearly drm.
This is what I would like substantiated. I can go to a site behind Anubis, download the content on it, and do whatever I want with it. It doesn't do any kind of attestation of hardware or software. So - what makes it DRM?
! Title: Hide Anubis Image
*/.within.website/x/cmd/anubis/static/img/*.webp$image
2) They're clearly talking from the perspective of the user and network admin. A residential proxy virus is in fact quite benign for both.
3) No matter what, DoS is quite a low priority in terms of security. And it's not even really DDoS, you'll know a real DDoS when you're on the receiving end.
That's only because it isn't being prosecuted appropriately.
I'm all for creating consequences for people who negligently don't secure their systems and people get hurt because of it, but people who installed a mobile game that has a proxy library in it? Please, chill.
I'd say yes since you've allowed random people unauthorized use of the connection. It makes no difference if it's your connection or not. For example, if that proxy gets used as part of a DDoS attack, your device compromised that network.
Yes. And someone did get unauthorized access to it. You don't have the permission to delegate authorization to a third party.
Did the third party connect to the guest wifi or did they connect to your computer and you have allowed them access to the guest wifi?
Did they agree to the ToS that the wifi network requires for people to use it?
Are you certain that the wifi network isn't intentionally limiting the area where it is broadcast, allowing only people within that geographic area the ability to connect to it?
If there is a purely 100% open wifi network who is only limited geographically due to technological constraints but would allow anyone in the world to connect to it if the technology allowed, then using your device to solve the technological constraint of limited wifi range would not fall under compromising the network IMO. Assuming again that the operator of the network publicized it as such and you aren't just making assumptions about their intentions.
There are two main differences. Netflix is working on your behalf. Connections initiated from netflix are no different from connections initiated from your web browser or your email client. Your use of the internet, for you, is authorized. With a proxy you have no idea who is connected or what they are doing with that connection. That network activity uses you, but isn't by you, or for you.
Finally, netflix isn't likely to be hacking, attacking, spamming, or attempting to bypass bans or IP blacklists over that connection, but those are exactly the kinds of activities that shady residential proxies are used for. By using those kinds of apps you basically backdoor/bot your own computer, and now access to the internet through it is being sold to people who want to use it for things that they couldn't get away with when using a VPN or any other legitimate provider.
Depending on the sophistication of the wifi network you may be held liable and accountable for everything that takes place over that connection, including whatever nasty things the people who bought access to your proxy get up to.
If the wifi system isn't set up to keep careful track of who is using it and when (as is common in places like coffee shops), the liability will fall to the entity offering the wifi access, and if that connection is repeatedly used for activity in violation of the law, or the AUP of the ISP being used, that same entity will face the repercussions. Their internet connection may be temporarily suspended. They may have to start blocking certain ports and services, or deny access to specific websites. They may even be forced to stop providing wifi access to guests entirely unless they are willing/able to spend the money needed to log and monitor who is using their connection and respond to notifications of abuse over their network as well as legal requests.
Places like coffee shops that just want to offer free wifi to their customers without worrying about that kind of thing are only able to do that because a small number of local individuals are using their connection. Locals who presumably enjoy being able to the internet at that location and won't take advantage by abusing it. Individuals who are also typically on camera, in full view of employees and other customers.
That kind of very common set up is insecure by nature and those companies are putting themselves at risk by offering the service, but it mostly works because the access is time/location-gated and most people in the neighborhood are generally not criminals. Allowing criminals from all over the globe to access those kind of connections through your device is allowing unauthorized access, and is also being a dick, since you're basically shitting in the pool everybody wants to be able to swim in. It greatly increases the risk that the wifi gets shut down, gets more locked down, or the IP address gets blacklisted making it less useful for everybody who uses it, including you.
I mean politicians are so addicted to twitter the nsa makes special phones…
It's always better to protect politicians from someone else