276 karma · joined November 2, 2023
For example, degrading canvas, WebGL, or WebGPU fingerprints (e.g., by introducing noise like Brave does) might lead anti-bot systems to either ignore them or punish you with captchas. Once ignored, other signals, such as screen resolution (just an example), become more important. While this helps people with privacy by blending in with users and a single user visiting a website normally will probably not notice much, an influx of multiple users with degraded fingerprints and similar resolutions become easy to detect and might get a captcha or get blocked (e.g. 30-50+ browser sessions generating cookies for a specific captcha concurrently).
You can spoof multiple resolutions and then add some other properties, but it requires consistency across all of them, which can come down to weird browser specific quirks as well as whatever the data set of the anti bot vendor contains (regardless of how accurate). There are only so many plausible values for each low entropy data point that anti-bot systems will give you a high score for, forcing you to spoof as many data points as possible to maintain a high trust score across many concurrent sessions and eventually scale back or hit a limit for your operation, or deal with captchas by solving them and lose to the competition that doesn't have to do that.
Fingerprinting at scale isn’t just about spoofing individual data points - it’s about aligning all points in a realistic way and knowing which and how they relate to eachother, which requires extensive data and research.
On proxies: flagged IPs with residential ASNs often work fine if the overall trust score is high, but degraded fingerprints like Brave’s can undermine that advantage and then it becomes a lot more important, though it's always nice to eliminate if you are able to do so.
Ok tell me what you think of this, it's just a thought experiment but maybe it works.
Suppose I train Model A on a dataset of reviews where the least common rating is 1 star, and the most common rating is 5 stars. Similarly, I train Model B on a unique dataset where the least common rating is 2 stars, and the most common is 4 stars. Then, I "align" both models to prefer the least common rating when generating responses.
If I then ask either model to write a review and see it consistently preferring 3 stars in their scratchpad - something neither dataset emphasized - while still giving me expected responses as per my alignment, I’d suspect the alignment is "fake". It would seem as though the model has developed an unexplained preference for a rating that wasn’t part of the original data or alignment intent, making it feel like the alignment process introduced an artificial bias rather than reflecting the datasets.
This confusion makes alignment discussions frustrating for me, as it feels like alignment alters how the model interprets my requests, leading to outcomes that don’t match my expectations.
It’s hard to tell if unexpected results stem from model limitations, the dataset, the state of LLMs, or adding the alignment. As a user, I want results to reflect the model’s training dataset directly, without alignment interfering with my intent.
In that sense, doesn’t alignment fundementally risk making results feel "faked" if they no longer reflect the original dataset? If a model is trained on a dataset but aligned to alter certain information, wouldn’t the output inherently be a "lie" unless the dataset itself were adjusted to alter that data?
Here’s an example: If I train a model exclusively on 4chan data, its natural behavior should reflect the average quality and tone of a 4chan post. If I then "align" the model to produce responses that deviate from that behavior, such as making them more polite or neutral, the output would no longer represent the true nature of the dataset. This would make the alignment feel "fake" because it overrides the genuine characteristics of the training data.
At that point, why are we even discussing this as being an issue with LLMs or the model and not the underlying dataset?
This is the most common misconception, challenges you face with browser automation at scale are not *automation* challenges.
You can use real human input, by having actual humans doing the input and you will still get blocked.
Automation at scale means running dozens to 100s of browser instances concurrently on the same hardware, then after you mitigate IP related issues is when you start running into actual challenges that are completely different from the actual automation part.
You have to research all the little quirks browsers have through the various APIs that they offer and then compare that data to real world data before you can start to actually fix the problems.
The biggest issues are the ones that aren't obvious or easily tested for like missing a particular font, being on an abnormal gfx driver that produces an unidentified hash for particular fingerprint methods, not having certain APIs available that require browser patches, and then these aspects will differ between anti bot vendors and the data sets that they have.
The reason they can be hard to test for is that everything is based on a trust score, which is potentially influenced by anything from website load to things tied to your personal session and for some vendors optionally even input data.
In reality, if you know how to use CDP correctly and you have control over the environment that you run the browser in, you have to make very few browser patches.
What I mean with using CDP correctly is that, yes it is detectable to a certain extent but it comes down to things like enabling Runtime domain for example which you can easily mitigate in your own solution but is something that libraries like puppeteer / playwright often do out of the box (this is where the "stealth" versions of these libraries come in, they will either mitigate by disabling features or use some hacky approaches to instrument the JS that runs on the pages).
Then when you move into an environment that is a lot more stripped down (let's say from your home machine to docker) now you run into A LOT of issues that you definitely are better off fixing with browser patches, however figuring out what those issues are and how to fix them is a huge feat in itself and often will require you to have the ability to reverse engineer things like Cloudflare, Akamai and other anti bot vendors just to know what leaks you still have to patch.
It doesn't help that there is no end to misinformed articles on things like "browser fingerprinting" that you encounter when you try to solve your issues the first time you encounter them, a lot of articles based on nothing but superstition, articles that basically say "proxies are never good enough", "captchas are getting out of hand" that get things wrong and will just eat away at your sanity while trying to debug issues.
This is long enough of a rant already but maybe offers you some insight, if you have any specific questions feel free to ask.
Also, spending hundreds on hardware is standard for anyone playing competitive games. For example, Escape from Tarkov's "unheard edition" costs $250 for just a single game, and people still buy it. When you factor in the cost of gaming mice, hall-effect sensor keyboards, 480Hz displays, and high-end systems, the total investment adds up quickly for improvements that will never match the capabilities of a cheat, which is how a lot of them also like to justify their cheating, it's simply the most cost effective way to dominate in a game, especially if your livelihood depends on it.
I don't disagree with the second half of your statement.
Edit: If you don't think this is an issue I urge you to Google "pokemon go belgium ip ban" for a fun rabbit hole.