Comment by ameshkov | Hacker News Reader

Comment by ameshkov | Hacker News Reader

Parent Full thread

Let me please chime in. As AdGuard developer, I'd like to comment on what you said and explain some details about how it works.

First of all, AdGuard desktop and mobile apps are quite different from hosts files or pi-hole. For instance, they're also able to apply cosmetic rules. Also, and this is really important on Android (unfortunately we can't do that on iOS), AdGuard is able to apply different rules depending on which app makes a request.

The common example is dealing Facebook Audience network. If you need to block Facebook ads in third-party apps with Pi-Hole, you'll have Facebook official apps broken as well. With AG, you can keep the official FB app working and block it in third-party apps at the same time.

Second, on every platform save for iOS, AdGuard filters every network connection and not just DNS queries. There're already multiple examples of apps (for instance, TikTok) that switch to using DOH when they detect that there's DNS filtering messing with their domains. Eventually, every network-level blocker, pi-hole included, will have to control all connections as DNS simply won't be enough.

Regarding Pi-Hole, we actually maintain a free and open source alternative called AdGuard Home: https://github.com/AdguardTeam/AdGuardHome

> For instance, they're also able to apply cosmetic rules.

How does this happen? Do you inject the thousands of CSS rules from EasyList in each page? Doesn't this require AdGuard to be able to have access to the response body for secure connections? Doesn't this mean that web pages can easily override injected cosmetic filters since these are not injected as user styles?

1. Yes, it basically injects a CSS stylesheet into every page + a JS script that does additional filtering.

2. Yes, in order for this to work, AG will need to access response body.

3. Yes, and in this case we'll need to use JS-based filtering (so-called extended CSS rules or scriptlets).

And here's an answer to the question you didn't ask: yes, this would be slower than doing the same with a browser extension. To make it faster, a different approach to filter rules is required - fewer generic cosmetic rules, more HTML filtering and removing elements from the page content.

The end goal is to completely get rid of any app or extension. Ideally, I'd like to see all the filtering moved to a server-side application like AdGuard Home.

Why not analyze the page and only apply the necessary generic rules, the way uBO does it? Am I missing something?

If all pages were static this would’ve worked beautifully.

But they aren’t, and uBO approach also implies that there is a mutation observer constantly monitoring DOM changes and adding new rules when they are needed. In a browser extension it uses browser’s own RPC to pass DOM nodes to the background page, check them, and apply new rules when needed.

In our case we would’ve needed to use something like a websocket to do it, but WS may be quite problematic. And the overall performance gain from this DOM-survey approach is milliseconds, it’s not something that a person can notice.

> AdGuard desktop and mobile apps are quite different from hosts files or pi-hole. For instance, they're also able to apply cosmetic rules.

My understanding is that AdGuard desktop is equivalent to hosts file + adblocking extension. Is that correct?

> AdGuard is able to apply different rules depending on which app makes a request.

How is this possible without root?

> AdGuard filters every network connection and not just DNS queries.

How can AdGuard filter secure connections (outside of the browser)? Does AdGuard filter by IP address?

No, AdGuard desktop is basically a full-scale firewall with a network driver that intercepts all network connections.

Regarding Android, this works with the help of the VPN API.

1. Android routes all IP packets to the “tun” interface

2. AdGuard reads them, passes through its own small tcp/ip stack. On one side there is the tun device, on the other side there are real sockets to the IP packets destinations.

3. App detection can be done either by reading /proc/net/tcp or, on newer Android versions, by using special getConnectionUid method.

Regarding secure connections, besides IP filtering (which AG also can do) there is always SNI scanning. Also, there’s an option to MITM connections, in this case AG generates a unique CA locally and does all the certs validation by itself.

I read more about how AdGuard works on the website (which is what I should have done in the first place, instead of making assumptions on how it works). I’m quite impressed, especially with Android, at how much control AdGuard has over network traffic. I’m left with a few questions.

- Many apps ignore the device certificates and instead use their own certs which come installed within the app itself (to prevent MITM attacks). How does AdGuard deal with this?

- iOS is much more restrictive. Other than Safari’s content-blocking API, does AdGuard for iOS only do DNS filtering?

- I’m not surprised that apps like TikTok are using DOH to circumvent filtering, but I can’t find a source online confirming this. Could you point me to an article/repo issue/etc. which confirms this practice and lists other apps that also do this?

1. SSL pinning is not actually that widespread. However, it is used by quite popular apps - Facebook and Twitter. Unfortunately, there is no way to deal with it without patching the apps itself. Also, modern Android versions limit the trust for user certs, basically only browsers trust that type of certs. The solution would be to move the cert to the system store, but it requires root.

2. It’s not iOS that’s restrictive but Apple. We can’t even mention anywhere in the app that you can block something using DNS filtering. We’re playing this reject-phonecall-reject-appeal game for two years already and I simply have no confidence that if we bring all the functionality to iOS they allow this. Other than that it’s possible and rather easy to do, the core filtering engine is implemented in C++ and we use it on all other platforms.

3. This is from talks with filters maintainers. But yeah, this is a good topic for an article, we should write one. Thanks for the tip:)

edit: my desire for moving filtering to the server-side actually comes from the experience of communicating with different stores. Also, Chrome’s upcoming changes contribute to my paranoia. Slowly, step by step, users are losing control over their own devices and apps.

Reply on news.ycombinator.com