ameshkov

Safari on macOS supports executing custom JS on web pages since forever, regardless of what version of API is used.

It’s not supported on iOS, though.

Note that YT changes aren’t yet rolled out everywhere. Also, if you’re not authorized there’ll be no issues, but it won’t stay like that forever.

Thank you for the very detailed explanation!

Proving IP address ownership is done via HTTP challenge, nothing too problematic.

So there would be ClientHello with greased ECH and SNI and ClientHello with real ECH and without SNI?

Regarding blocking, what prevents Chinese firewall from simply removing ECH extension from all ClientHello packets? Servers that don’t expect ECH would continue to work, those that expect wouldn’t accept connection, mission accomplished.

If we're keeping this simple, then I'd say have IP address in subaltnames. Obtaining such certs shouldn't be a problem for CDNs, and possible for others as well (regarding free options: Let's Encrypt doesn't support that, but ZeroSSL does)

As someone suggested in a different comment: handshake with one certificate, then send a new ClientHello that contains the desired hostname and "re-handshake" with the real cert?

I've been loosely following ESNI/ECH drafts, and I am not sure this approach (double handshake) was ever seriously considered despite that it sounds perfectly fine to me. It'd make the handshake heavier, but on the other hand, it'd also make it much harder to distinguish from the regular TLS traffic. On the contrary, ESNI/ECH seems to be easy to detect and block, and some countries (China, for instance) have already announced that they're going to do that.

> it appears they assume people are using DoH or DoT

Things changed since then, now plain DNS is also allowed.

Can anyone explain why is it designed this way? Why was it necessary to involve DNS into this? Was it unavoidable or is that all in the name of keeping 0-rtt possible?

Tbh, with the current implementation, ECH setup seems rather complicated to me. It benefits CDNs like CloudFlare since they control both DNS and the handshake process, but those who would like to setup ECH on their own are in trouble.

AdGuard dev here. Can confirm - it’s not sufficient. But the reason it got less criticism is a little different. Content blocking in Safari was never good, and content blocking API when it just appeared was a move forward. The problem is that it’s been five years and it hasn’t got any viable improvements. Manifest V3 in its current state is already way ahead of Safari. But how good will it be 5 or 10 years from now?

All free software that we make is open source. Paid - closed source (save for AdGuard for iOS, it's open source despite having some premium features).

This just makes me extremely sad.

Server products is what made us purchase Atlassian software. There are numerous reasons why our company (and a lot of other companies I presume) would like to avoid cloud. And if we really had to go cloud, I simply don't see why we would choose Atlassian over alternatives.

1. SSL pinning is not actually that widespread. However, it is used by quite popular apps - Facebook and Twitter. Unfortunately, there is no way to deal with it without patching the apps itself. Also, modern Android versions limit the trust for user certs, basically only browsers trust that type of certs. The solution would be to move the cert to the system store, but it requires root.

2. It’s not iOS that’s restrictive but Apple. We can’t even mention anywhere in the app that you can block something using DNS filtering. We’re playing this reject-phonecall-reject-appeal game for two years already and I simply have no confidence that if we bring all the functionality to iOS they allow this. Other than that it’s possible and rather easy to do, the core filtering engine is implemented in C++ and we use it on all other platforms.

3. This is from talks with filters maintainers. But yeah, this is a good topic for an article, we should write one. Thanks for the tip:)

edit: my desire for moving filtering to the server-side actually comes from the experience of communicating with different stores. Also, Chrome’s upcoming changes contribute to my paranoia. Slowly, step by step, users are losing control over their own devices and apps.

If all pages were static this would’ve worked beautifully.

But they aren’t, and uBO approach also implies that there is a mutation observer constantly monitoring DOM changes and adding new rules when they are needed. In a browser extension it uses browser’s own RPC to pass DOM nodes to the background page, check them, and apply new rules when needed.

In our case we would’ve needed to use something like a websocket to do it, but WS may be quite problematic. And the overall performance gain from this DOM-survey approach is milliseconds, it’s not something that a person can notice.

1. Yes, it basically injects a CSS stylesheet into every page + a JS script that does additional filtering.

2. Yes, in order for this to work, AG will need to access response body.

3. Yes, and in this case we'll need to use JS-based filtering (so-called extended CSS rules or scriptlets).

And here's an answer to the question you didn't ask: yes, this would be slower than doing the same with a browser extension. To make it faster, a different approach to filter rules is required - fewer generic cosmetic rules, more HTML filtering and removing elements from the page content.

The end goal is to completely get rid of any app or extension. Ideally, I'd like to see all the filtering moved to a server-side application like AdGuard Home.

Just a quick note about that:

> does not have to wait for a DNS resolver to respond

The browser (or the system) caches DNS responses and the query for a blocked domain won't be repeated for quite some time.

Well, to be honest, AdGuard not-the-extension technically cannot be the fastest. It does a lot of things to do system-wide blocking - analyzing the connection, passing network packets, parsing protocols, this all adds some overhead. Browser extensions simply don't deal with all that stuff and thus any extension, unless it's really poorly written, is not slower.

Moreover, this is not what we are offering. System-wide blocking is about having more control over your system and not about being the fastest.

No, AdGuard desktop is basically a full-scale firewall with a network driver that intercepts all network connections.

Regarding Android, this works with the help of the VPN API.

1. Android routes all IP packets to the “tun” interface

2. AdGuard reads them, passes through its own small tcp/ip stack. On one side there is the tun device, on the other side there are real sockets to the IP packets destinations.

3. App detection can be done either by reading /proc/net/tcp or, on newer Android versions, by using special getConnectionUid method.

Regarding secure connections, besides IP filtering (which AG also can do) there is always SNI scanning. Also, there’s an option to MITM connections, in this case AG generates a unique CA locally and does all the certs validation by itself.

Let me please chime in. As AdGuard developer, I'd like to comment on what you said and explain some details about how it works.

First of all, AdGuard desktop and mobile apps are quite different from hosts files or pi-hole. For instance, they're also able to apply cosmetic rules. Also, and this is really important on Android (unfortunately we can't do that on iOS), AdGuard is able to apply different rules depending on which app makes a request.

The common example is dealing Facebook Audience network. If you need to block Facebook ads in third-party apps with Pi-Hole, you'll have Facebook official apps broken as well. With AG, you can keep the official FB app working and block it in third-party apps at the same time.

Second, on every platform save for iOS, AdGuard filters every network connection and not just DNS queries. There're already multiple examples of apps (for instance, TikTok) that switch to using DOH when they detect that there's DNS filtering messing with their domains. Eventually, every network-level blocker, pi-hole included, will have to control all connections as DNS simply won't be enough.

Regarding Pi-Hole, we actually maintain a free and open source alternative called AdGuard Home: https://github.com/AdguardTeam/AdGuardHome

This may be useful to you: https://github.com/adguardteam/hostlistcompiler

There is another interesting paper on a similar topic that proves that this approach is vulnerable to different adversarial attacks, for instance, you can make it block legitimate content, or you can conceal ads by adding an overlay invisible to a human eye.

https://arxiv.org/abs/1811.03194

Acceptable ads deserves being there, you are right.

This was the first attempt at finding a compromise with advertisers, hated by a huge part of users and advertisers at the same time. But it was launched by Adblock Plus in 2011, long before uBlock was developed.

1. The only debugging tool we have is the browser console, and the only thing printed there is the blocked URL, you cannot even find out what rule did that.

2. The maximum number of rules IS a problem and it is there only because the current implementation consumes to much memory and slow to compile a content blocker. WebKit devs may allow us to pull request an alternative implementation (there’s an ongoing discussion on bwo), and if there are no performance issues, the limit will be increased.

3. Regarding maintaining a filter list, traditional blockers do not use regexes unless it is really really necessary. What’s written in this post is by people who maintain filter lists for over 10 years.

Sorry for that, we don't mind mentioning anyone, it's just there was no historical event that we could tie to uBO. I'll ask the author to add a footnote about uBO and others we missed to mention, we love them all.

upd: done

Shadow DOM is created dynamically using "attachShadow". You can try overriding Element.prototype.attachShadow, but this is easy to detect.

Actually, Web Components and Shadow DOM bother me more than Web Assembly. You cannot "pierce" into a shadow DOM from the outside and this creates an impenetrable barrier for content blockers. We've already seen examples of whole web pages transformed into a shadow-dom web component.

Not directly related to the topic but anyway, can anyone explain how come Apple rule (every browser has to use WebKit under the hood) was never openly disputed? I am pretty sure Mozilla, Google, and others would love to bring their engines.

Actually, they are quite helpful on the developer forums.

Fully agree on the documentation point, though.

Well, let me disagree. The documentation is lagging. Right after the WWDC there were nothing except for some header files.

Also, why half a year? Do you think it will take them that much to release 10.15.4?