How to take back control of /etc/resolv.conf on Linux (2018)
ctrl.blog
ctrl.blog
I ended up writing this series to exorcise the demons:
https://zwischenzugs.com/2018/06/08/anatomy-of-a-linux-dns-l...
https://zwischenzugs.com/2018/06/18/anatomy-of-a-linux-dns-l...
https://zwischenzugs.com/2018/07/06/anatomy-of-a-linux-dns-l...
https://zwischenzugs.com/2018/08/06/anatomy-of-a-linux-dns-l...
Personally, I've set adsuck and my /etc/resolv.conf points to 127.0.0.1
Haha this is what I do…
NetworkManager uses dnsmasq or systemd-resolved as it's backend...
I’ve suspected for a while, and am now pretty sure, that this system has become so byzantine that the (incredibly talented) people assembling distros, writing the software, and maintaining the man pages, don’t know how it works.
Which is... precisely what a lot of people were worried about re: systemd :(
If it were just resolved or just the other tools doing wacky things DNS stuff would be a lot less messy to configure.
If you think things are byzantine with systemd, you should have seen how things were before!
The old stuff was byzantine, but it was decades old, which meant that it had decades of people beating on it, figuring out all the ways that it could flake, and fixing those flakes. And that's not something to sneeze at.
That doesn't make the old stuff any less byzantine. And systemd really is a lot more coherent than most of what it replaced. But systemd doesn't have decades of people finding all the ways it can fall over.
What I’ve enjoyed in particular: systemd timers are an improvement over cron; and I’ve been making great use of systemd-nspawn, which is very convenient and powerful.
there were plenty of growing pains at first, but the transition period for any project looking to replace the old init system was going to be a headache no matter what. It does take some time to grok, and it feels monolithic at first, but I actually think that's a good thing. It's composed of many separate binaries that individually conform to the unix philosophy, but they're all distributed, maintained, and released together. You still have tons of configurations options, but you can be confident that standard systemd components are always going to play nicely with each other.
The only issue I've really had are the binary logs. If the indexes get messed up it's a pain to access the logs even if though the actual files rarely have issues with corruption. But it's fairly easy to forward messages to syslog if you need.
Exactly as it should.
> This causes all sorts of issues with private DNS servers on lans, VPCs, and VPNs where you want a failover to a public DNS while resolving.
This is broken; you should have per-interface defined DNS. For internal lans, vpns et cetera you can define which zones these DNS servers handle. For public resolvers (designated with handling the ~. zone), just having multiple entries provides the fallback you want.
The main missing concept in the article is DHCP. Most computers use this to get their IP and the DNS servers. For example, for as long as I can remember the default Debian way to handle the network was ifupdown, where a hook called `dhcp` after an interface is connected, and by doing so the file resolv.conf is updated. If resolv.conf is manually controlled, there is no DHCP to update it, and then you have to declare your DNS servers. But the DNS that you can query from home may not be reachable from a public wifi. And at work, you won't have access to the intranet unless you add some private DNS servers at the top of your resolv.conf. So a totally manual handling of resolv.conf is a bad idea, at least for a laptop.
The article also never wonder why do all these programs try to "take control" of resolv.conf. Some of them just try to update it when they receive DHCP data about DNS. Other, like resolvconf, try to separate the manual part from the dhcp-generated part (which is what "manual control" should aim to). Other programs may add features, like a local cache or automatic knowledge of the local network (containers and such).
I use OpenBSD on that machine. I think it was "ignore domain-name-servers" in dhclient.conf that fixed that for me.
Careful. Even if you point at a different dns resolver, your ISP still gets to see this traffic, and it can still MITM it. This is not theoretical, it is trivial. Many ISPs do it.
Some people use e.g. opendns because they want to use that for keeping track of what dns names are being queried from inside their networks.
I actually do use my ISP's dns for some names (particularly netflix), because only they know the names of their internal cache boxes. I have a dnsmasq configuration that sends queries that end in `netflix.com` to the ISP dns and others to 8.8.8.8/1.1.1.1 or whatever.
Oh it looks like good setup, I'll try it.
* https://news.ycombinator.com/item?id=24280536
I've seen no evidence that Virgin Media intercepts UCP/TCP port 53 traffic, however. It just runs rigged proxy DNS servers and farms them out over DHCP.
I've seen several reports that the Virgin Media "click here to disable" mechanism is a placebo that actually does nothing at all, including what that message was a reply to.
The OpenBSD box has its own dhcpd and dns server. The leases it offers point to itself for dns in the LAN. Then it makes forwarded queries over TLS.
https://man.openbsd.org/unwind.conf lists some Do-T-related config options.
Totally off topic, but I was thinking of replacing it with something I wrote myself. I had some downtime earlier this year and I wrote a forwarding dns server that can speak TLS on both ends. Was simple to do. But needs some polishing before I throw it on github or use it at home.
(YMMV again with trust paths but eyes open helps)
On the other hand, DNS over HTTPS defeats ISP DNS interception regardless of who supports it, which is why so many more people use it than use DNSSEC, which is moribund.
First of all, /etc/nsswitch.conf set the priorities. For instance the line `hosts: files myhostname mymachines dns` would exclude systemd-resolved (aliased to `resolve`) and try to identify containers before querying DNS. See the man.
I used to use resolvconf in order to merge my local rules with those set automatically by DHCP. Nowadays I have a dnsmasq daemon running a dns proxy. This way I can set rules so that some domains are linked to special DNS (e.g. *.debian.org should be resolved through 1.1.1.1 instead of the default that the DHCP set). And added benefit is the filtering of ad domains.
All the tools this article talks about turning off are trying to solve problems. They're trying to let resolv.conf be configured with DHCP, or handle VPNs correctly, or roam across networks. You need software to do that, not just a static config file. The problem is every Linux distro has a different tool. And those tools behave opaquely or do bizarre things like use 127.0.0.53 as the loopback address. No one built the One True Tool that got it right. And so we have a mess that has the sysadmin throwing up their hands, disabling all the tools, and making immutable config files just to get back to basic static file functionality.
Finding the One True Tool takes time. Developers need to experiment to explore the various corners of the problem space. Eventually we'll settle on the best way to solve the problem for the majority of use cases, and distributions will in time settle on that solution.
In the meantime, sysadmins who only need the simple 20 year old use case can continue to disable the modern tooling and get what they want. Those who need the newer use cases can do so, provide feedback, and the ecosystem can slowly develop and settle on the best solution.
This is open development of Free Software in action. Tools can evolve incrementally or new tools can take big leaps, taking advantage of both the lessons learnt and the code produced in older tools. It's how the ecosystem gets better. I don't see it as a condemnation at all.
not systemd. gah!
* http://cr.yp.to/dnsroot.html
* http://jdebp.uk./Softwares/nosh/guide/services/djbdns.html
You don't erroneously think that the systemd people invented this idea, do you? It's in RFC 1122.
I’m not a big fan of sysadmins coasting on their existing knowledge, and this sentiment sums up what I object to precisely. Complexity has increased somewhat since the Slackware days, and it doesn’t always seem to be for the best, but that’s no excuse for disabling it.
Expressing this viewpoint in an interview would materially harm a sysadmin candidate’s chances: it suggests that they are inflexible, unable to adapt to change, and prefer to simply sweep issues under a rug rather than document them and resolve them.
I empathize with the desire to set it all aside and go back to bare metal, but it’s not a position I would take as much pride in.
Seriously, this is the best possible reason to disable something. You don't just submit to downgrades because progress is progress.
The problem is they all think they did.
chattr +i /etc/resolv.conf
To make it immutable.Also I'm not necessarily saying it's a good or a bad solution. But it works, so there's that.
It relies on /etc/resolv.conf being a symbolic link to it's generated version in /run/systemd/resolve/resolv.conf to do its thing. If it isn't linked, it assumes that is intentional and leaves your resolv.conf alone.
> To make the resolver use this dynamically generated resolver configuration file the administrator should ensure that /etc/resolv.conf is a symbolic link to /run/resolvconf/resolv.conf. This link is normally created on installation of the resolvconf package. The link is never modified by the resolvconf program itself.[0]
> To improve compatibility, /etc/resolv.conf is read in order to discover configured system DNS servers, but only if it is not a symlink... [1]
[0]: http://manpages.ubuntu.com/manpages/zesty/man8/resolvconf.8....
[1]: https://www.freedesktop.org/software/systemd/man/systemd-res...
Makes me wonder if the file options should be extended contain a config option to indicate what program manages the file so that others know to keep their hands off. Right now there's comments in there, but those comments are often hardly machine readable.
No, we need to get back to software like it was in the early 2000s when people cared about consistent, long-term behavior instead of re-inventing everything because laptops.
If you want them to have broken DNS, then they are fine. If you want them to have working DNS, then not, regular static file is not fine.
If you only let one thing on your system own it: NetworkManager, dhcpcd, systemd-resolved, netplan, ifcfg, $you then it’s no issue.
$ cat /etc/resolv.conf
# DO NOT EDIT THIS FILE. This file is currently under the management
# of NetworkManager. It's automatically generated according to the
# DNS configuration of the active network connection in NetworkManager.
# Any change to this file will be overwritten.
#
# If you are trying to change your DNS resolver, it's officially recommended
# to set it within NetworkManager on a per-connection basis via "nmtui", "nmcli",
# or alternatively, by editing files under "/etc/NetworkManager/system-connections/".
#
# To stop NetworkManager from managing it altogether, edit
# /etc/NetworkManager.conf, set the following and
# restart NetworkManager, before you edit /etc/resolve.conf.
#
# [main]
# dns=none
#
# If you are trying to set up a stub DNS resolver, it's recommended to
# invoke NetworkManager's builtin support. Set "dns=dnsmasq" or "dns=unbound"
#
# See "man 5 NetworkManager.conf" for more details.The problem with RFC 1034 section 5.3.1 ideas is, that it is insufficient if you have more than 1 interface and DNS resolvers available over them might provide different answers for some zones (example: you have a VPN connection and the DNS resolver on the other side of the link has ACLs for the zones it is handling, providing you with answers that the public DNS won't provide); that's why networkmanger+dnsmasq/systemd-resolved/whatever are used.
Linux is not the only system that has to handle /etc/resolv.conf being inadequte; but it is funny that we see articles like this only for Linux. I have yet to see an article "How to take back control of /etc/resolv.conf on MacOS".
It took me an hour of reading the ubuntu official documentation and a few man pages on systemd, interfaces, networkd, resolvd, netplan etc to figure out how stuff works on this distro.
After that I was able to simply do 'systemctl status' and look at all the running processes, then stop and disable all the ones I didn't want.
I wanted to run unbound dns as my dnssec verifying, tls-forwarding, blocklist/filtering resolver. I was able to do that without any difficulty – stopped resolvd, enabled and started unbound and changed the entry in resolv.conf. I also brought up dhcpd for my network and was able to serve that dns server to the devices on my network.
Then I had to spend a bit of time figuring out pppd, policy routing and nftables to do a bit of fancy dual wan routing and traffic filtering. I found netplan to be quite good at what it does (still incomplete for more advanced scenarios - but then so was all the previous tools similar to this).
In all this, I actually found systemd quite helpful. I think those who complain don't take the few minutes it takes to read the man pages and familiarize themselves with the tools on the OS.
Ofcourse things change, and you will have to learn new things. Expecting things to stay constant is unrealistic. As long as I'm able to depend on man-pages and official distro documentation, I'm not objecting to change.
Not to mention the fact that the information containing the proper steps to do this simple task of often buried among other network configuration information.
I think SUSE had the best approach: if file doesn’t have a certain header (a comment stating something like “automatically created by x”) it means management scripts shouldn’t touch it. Not sure if it’s still like that.
https://bugs.launchpad.net/bugs/1624320
My hate for this whole situation (but especially the fact that it broke resolution of short names with search domains) is bone-deep.
"The primary purpose of adding 127.0.0.53 to resolv.conf is for client software that wants to do DNS resolution by itself instead of using NSS -- most notable example is Google Chrome, and third-party software which is statically linked (e. g. Go)."
All this mess just to work around Chrome? That's annoying.
Sure there's a little more work to get set up (which I've scripted and put in Gitlab, so it's easy the second and subsequent times).
But the documentation is first class (in particular, the FreeBSD Handbook is great), and generally, when you set something up it stays that way.
Reminds me of late 90s / early 2000s GNU/Linux in that regard.
OpenBSD would be even cleaner and leaner, but I needed wine, keras, theano and a few other things it didn't offer.
I'm having a few issues with the docking station on the W540, which I'm currently putting down to the docking station being faulty, as I had no such issues docking the X220.
I plan to test a different docking station in the next few days and see if the glitches go away (USB mouse sometimes needs resetting, and X loses the external monitor until it's restarted).
This seems similar to the "don't disable selinux, learn how to use it" shenanigans..
The popular TLD nameservers would receive a lot of traffic but the amount depends on average TTLs in the wild — metrics which hopefully someone here will be knowledgeable about.
Presumably handling the traffic is nothing compared to handling www.google.com, but the TLDs can’t afford the same scale of infra that FAANG can?
Several billion people resolving their favourite websites every 86400 seconds — it’s either a lot, or one of those things the C10k folks love to show can be handled by a Pentiun III.
https://news.ycombinator.com/item?id=24231857 https://news.ycombinator.com/item?id=24270369
But if you want to add a TLS handshake to each request and that sort of thing, not so much.
sudo apt install resolvconf;
sudo vi /etc/resolvconf/resolv.conf.d/head;
sudo service resolvconf restartI installed pihole one a server once instead of running the image. It was amazing to see the 3 way fist fight that broke out on resolv.conf
I give it to all members of the wider family that I convert to Linux and I have not had any tech support requests related to that ever.
You can never answer that question with 100% certainty with "yes", only with "no".