Ad blocking with Raspberry Pi and Pi-hole

The strings are special structures which allow binary data to be stored in them and to protect against buffer overflow protection, so they take up more room than a normal string.

Since the cache is a dynamic LRU cache (blacklist entries are not in the circular LRU, of course, but the cache was designed that way) for temporarily storing correct DNS answers (with overhead because, yes, each entry has among other things a 64-bit expire timestamp), each entry is created or removed with multiple malloc() and free() calls, so there’s some page size overhead there.

Like I said, there are ways to make a blacklist have a smaller memory footprint, by using a special data structure which trades speed for size, [2] but for a two-day coding project, it made more sense to just add them to the speed optimized LRU cache the code is built around.

[1] As an aside, the hash compression algorithm came out before SIP Hash, but, like SIP Hash, has protection against an attacker finding hash bucket collisions. If I were to write the code today, I would use SIP Hash.

[2] Here is how I would do it: I would have two blocks of data. One block is an array of offsets (pointers, but not C memory pointers) to data in the second block, and the second block is a bunch of NULL-terminated strings compactly stored. The array of offsets is a sorted list, where the sort key is the string each element points to. I would then do a binary search every time I wanted to see if an element is blacklisted. Memory overhead is, for 10 million entries with an average length of 15 bytes per string (including the NULL) 30 million bytes for the offsets and 150 million bytes for the strings, so we could store 10 million entries in under 200 megs. Speed is O(log2(N)) instead of O(1) but we save a lot of memory. We would need more memory to sort that list, but we could store this block as a binary file which is made on a host with a lot of memory, but read on our Raspberry Pi or old Netbook. Again, doing this is left as an exercise for the reader, and I’m sure there’s a lot of public domain or otherwise open licensed libraries which have already solved this problem. If I were to go down that path; I would probably use sqlite, but am a bit reluctant to add sqlite to a tiny program which can compile to be under 70k in size.

    upstream_servers = {}
    upstream_servers["."] = "192.168.253.253" # Never answers
    upstream_servers["good-domain.example.com."] = "9.9.9.9"
    upstream_servers["whitelist-entry.foo."] = "9.9.9.9"
    # And so on

But the bottom line is that the number of attempts to resolve the DNS doesn't say anything about how egregious the activity is. You could have a device try every second because it wants to send an "I'm alive" message home, or you could have a device attempt DNS resolution once per day and send all the traffic it captured from your network. Don't use that number as an indication of how evil the device is, I'm sure the most egregious malware or crappy IoT will be careful not to flood DNS requests.

But I have an instance that I already use for other things (IRC bouncer, Mumble server, Taiga task/issue tracker, and test server for a game I was writing) that has plenty of CPU time to spare, so it's not costing me anything.

But for the most part, I imagine it won't make much of a difference. I live within 300 miles of the data center containing my EC2 instance.

I have no idea if the Pi-hole is configured with this extension.

The only other way would be MITM all https traffic and sign with your own certificate, but many iot devices won’t allow you to install a new certificate store.

They would still have to know that the site isn't working because of blocking, but I suppose developing the habit of "website doesn't work, press the magic button and try again" isn't so bad.

If I do PiHole just for myself I think I'll add this button. Thanks!

Works great, even if I dont invoke Siri to trigger it.

Usually sites where the dev has based functionality on a JS module loaded from an advertise/tracker site, and that is being blocked, resulting in missing functionality.

Two prominent examples that I've personally had to deal with: CVS, and Taco Bell's iOS app.

Edit: Oh, and google's inserted redirects in shopping results lists are blocked by default.

That comes up now and again, not as much as you'd expect but enough that I needed to give instruction on it. Funny thing is, most of those cases are _me_ doing things like signing into Nvidia's software instead of anyone else here.

It should never be considered an "or" situation; you want "both" as much as possible.

I would love to see real data on this! Most of my peers don't want to be marketed to, at least when it comes to internet advertising. Definitely when it comes to traditional advertising (ie billboards are an eyesore, TV ads are offensive or irrelevant or just plain annoying).

For my Pi I regularly update without a reboot, but that might not be enough for kernel updates. I will look more into it.

As I read your comment all I could visualize was an IE6 browser with 8 toolbars. The network stack of the future.

-- mostly: out of about 7 videos, an ad was shown to me at the start of one of them, but that is much better than mobile Safari, on which and ad is shown to me at the start of every video I start.

Thanks for the tip.

I like how you phrased this as a solved engineering problem.

Although with DoH these days, I'm not confident my firewall rule is still doing a good job :(

I don't want to add an extra router because that would add unnecessary latency. The above is not an unusual setup at all.