245 karma · joined March 24, 2022
Except it won't impact gmail unless you're accessing it through curl. The browsers already distrusted the compromised intermediates via CRL within hours of disclosure, and removed the affected root via update.
So this isn't like Firesheep [0]. Yes, there would be vulnerable traffic, like the git client this thread is discusing, but it's not "everything"
These services cost money. Scrapers also cost money in terms of resource utilization and downtime. There's always a tradeoff.
The ROI of services like Cloudflare bot protection, Datadome, HUMAN, AWS bot mitigation etc. is highly dependent on your income from the data. In the case of publicly hosted freely accessible data it can be hard to justify.
Is this not what CAA records were intended to solve? It seems like in practice they're only used by CAs to decide if they can sign a cert for a given domain, but it seems like a client could use it to determine if a server cert was from the correct CA.
Oh no! They'll stay a buggy mess. Damn, I was really hoping the windows app might get cleaned up.
But beyond that, what are you referring to as compression? When I look at the source I see a VERY short easily human-readable HTML page. It doesn't have newlines, but I wouldn't consider that compression. It also references a short, but non-obfuscated javascript file at https://example.com/s.js .
Maybe I misunderstood what you're referring to...
The thread you linked to suggests 'timeout'[0]
[0] https://learn.microsoft.com/en-us/windows-server/administrat...
For behavioral, we have Anubis honeypot functionality turned on, but it doesn't seem to be effective for 99% of scrapers. Anubis is also running behind TLS termination, so I don't think it can do full JA4. It does have the less robust JA4H apparently, but I'm not sure how effective that will be.
Edit: Oh yeah, forgot to mention - it's almost 100% residential proxies. Primarily China Telecom and China Unicom. Unfortunately those providers are HUGE and also host a ton of legitimate users all over Asia.