Inside Firefox’s DNS-over-HTTPS engine
daniel.haxx.se
daniel.haxx.se
That's a total violation of the standard and will break A LOT of things. Example: my.domain.com -> CNAME ec2-1-2-3-4.aws.com 30s TTL -> A 1.2.3.4 30days TTL.
So Firefox will now cache my.domain.com to 1.2.3.4 for 30 days? When you update the record for my.domain.com the change today will be applied in 30s, but with this flawed heuristic it won't expire until after 30 days.
I mean, do we really want all of our desktop applications to have their own built in custom ways of mapping domain names to IP addresses?
[edit] E.g on Linux, it could install an application with a DNS interface listening on localhost port 53, which would then convert the request into a "DNS over HTTPS" request, and resolv.conf would be updated to use that resolver.
And, like you said, this is a bad idea. Is there something wrong with efforts like DNSCrypt + DNSSec? That's supposed to provide authentication and encryption to DNS without sending everything over HTTP.
Did Mozilla just totally ignore the work that's already been done in this area?
Every operating system has a system wide way of resolving names to IP addresses, and every application uses it. This new architecture of building custom name resolution into individual applications might be easier for them to build, but it's crap.
Mozilla is doing two things:
1. Bundling DNS with the browser (e.g. ignoring system DNS)
2. Using DNS over HTTPS
Mozilla could still do (1) and then use DNSCrypt + DNSSEC internally. Then, it would actually be used, but they'd be relying on existing technology that actually fits the use-case, rather than DNS-over-HTTPS.
For the record, I don't think you should ignore the system's DNS, either.
DNSCrypt is a niche protocol that basically does do what DoH does. Very few people use it. You can use it instead of DoH, if you like; just disable DoH in Firefox and set up DNSCrypt with your system resolver.
However I disagree that it is a bad idea and that the implementation is bad. Regardless of how software _should_ behave, Firefox operates in how software is actually run for their users. DNS is a source of security vulnerabilities and headaches.
Demanding a higher level abstraction is not always an option for many, but using Firefox often is. This is especially important for mobile, where a lot of people don't have access or knowledge to set in place a system wide proxy after rooting their phones, but it is very easy to install Firefox mobile.
What about web browser usage on library or campus computers? Often they will have several browsers installed as well.
The point is that making security more available and easier to use where it matters most is a good idea.
If you know of a DNS over HTTPS client for Windows, please link it!
TRR doesn't read or care about /etc/hosts
There's no way to exclude or white list specific domains"
For me, the primary advantage of HOSTS/DNS is the ability to control answers to application queries for addresses and block ads.
This seems to remove all control a user might have through controlling such lookups. Yikes.
I think DOH is useful but in a different way. For example, it is useful for retrieving bulk DNS data using RFC 2616 pipelining, alleviating dependence on piecemeal DNS lookups, thus increasing speed and privacy. Data can be stored locally and refreshed periodically, if necessary (I have been doing this witout problems for 15 years). It's also useful for retrieving data from a variety of caches, allowing answers to be compared.
[1] https://github.com/st3fan/tinydoh [2] https://github.com/m13253/dns-over-https
To put this another way: it's a significant benefit for my random non-techie friends to be able to use this new feature plus HTTPSEverywhere. And even as a techie, I don't use a hosts file to block anything. So I'm not bothered by how Firefox chose to implement this, like most people.
That kind of thinking, really irks me.
It does bother me when Firefox introduces a feature which covers up a huge hole in TLS, and they get a large number of complaints on HN.
Mainstream users do not need most software ever made.
Sigh. This is aggressively breaking normal DNS behavior (and will be an absurd hassle for a very large number of organizations, but in terms of extremely normal split-horizon and orgs with regulatory obligations to catch HTTPS traffic).
Applications should not contain their own encapsulated resolvers, let alone resolvers that default to sending all of my DNS traffic to for-profit companies that have previously experienced massive data leaks (and fun CF fact, they invited the then-CTO of Cambridge Analytica to talk at their Internet Summit event in SF last year).
they can disable it, any organization that modify /etc/hosts can also change Firefox's preferences file
In general it's not acceptable to break functionality and then demand people invent workarounds. But it's insane to demand that every organization in the world write new portable system integration software that has to take into account 100 varying things just to disable something nobody has asked for. And it's even more insane when the software in question is the underpinning of all internet access that has existed in the same form for 40 years.
And I trust Cloudflare, and more specifically Mozilla legal's ability to negotiate with Cloudflare, more than I trust approximately any ISP.
You don't have to trust anyone, you can run a recursive resolver too. Even some home routers do that already.
Chrome in particular has a DLL blacklist that seems to work very well in practice: in theory, a DLL blacklist can't possibly work, but in practice, DLLs don't try hard enough to hide their existence from the blacklisting code. https://www.chromium.org/Home/third-party-developers#TOC-Goo...
Not everyone wants the thing I want. But that's different from nobody wanting it.
(However, a system-wide DNS resolver using DNS-over-HTTPS is definitely a thing I want! I've been considering writing an NSS module in Rust for it, as a way to play with writing a loadable module with tokio, to see if that even works.)
Two, running a proxy to move all my browsing elsewhere seems like the hack. Changing the internet's norms so normal DNS is HTTPS-based, and everything just works everywhere for everyone, seems like a stable long-term solution.
So I had a DNS server that used UDP. Now I need a DNS server and an HTTPS server that uses TCP. Now I'm encapsulating DNS in HTTPS. So now the packets are bigger and there's extra latency, increasing traffic and slowing things down. (These kind of "fixes" must be common, because I seem to notice how technology is constantly getting slower and crappier rather than the opposite)
Then there's security concerns, because you're running twice the software with greater complexity, yet it's subject to the same attack methods as everything else, meaning one exploit could impact DNS just as much as every other HTTPS service. Among these are attacks on the PKI system, which is fragile to say the least - an expired or revoked cert kills your DNS, and any CA can generate a cert (which apparently isn't hard to do using a BGP exploit) which can be used to circumvent the encryption. Since you need DNS to do anything on the net, it will be trivial to simply kill all DNS-over-HTTPS queries going over a network and force people to use a downgraded, insecure version of DNS. And you've got to handle encryption in a way that is normally for streams rather than datagrams - unnecessary for data which only really needs authenticity and integrity assurances (which is, again, already provided by DNSSEC)
Then there's reliability. You're tunneling DNS over a connection to one server. We already know this is slower, and we already know DNS is one of the factors that slows down page connections. So you had better hope that the one HTTPS endpoint you have a persistent pipelined connection to doesn't go down, or wait and wait for what you hope is an independent 3rd party's service that is also in your list of DNS-over-HTTPS resolvers to pick up the slack.
As a DNS admin, I had software that communicated with DNS servers. But now I have to rewrite everything to use DNS-over-HTTPS, because if a query fails in DNS-over-HTTPS, you have to troubleshoot the actual connection and not what you think is in DNS, or what you think DNS should be sending back to the client. So now I have to reinvent every DNS tool. And of course, for the enterprise DNS solutions that can't just magically sprout a new protocol extension, I'll need some sort of middleware software to map the DNS-over-HTTPS queries to the real DNS server.
All of this because someone wanted "privacy" of their DNS queries, and was too lazy to allow UDP packets greater than 512 bytes over their network.
DoH provides confidentiality and query integrity now, without requiring a boil-the-ocean step where every DNS administrator on the Internet signs their records.
Solving this is known as "the robustness principle," aka "internet engineering." Expecting every device in between every two points on the internet to a) work reliably and b) not make questionable decisions about packets that it can parse has never worked. Successful internet protocols, e.g. TCP and HTTPS, are those which put the least reliance on the network to behave reasonably. Unsuccessful ones, e.g. IPv6, are the ones that put the most reliance on the network to behave reasonably.
It's true that the successful ones are nowhere as pretty as the unsuccessful ones. But for most people, success is more important.
The willingness of people to put all their stock in a corporation with a promise of benign benevolence boggles my mind. I'm not particularly cynical, but I recoil at the notion that you'll now need an entire HTTP+TLS stack just to do DNS "properly", and that 99% of people will be using a single DNS provider.
EDIT: It's not even benign benevolence. I see that DOH includes server push. So now it's obvious how CloudFlare will monetize this--they're positioned to be able to push DNS updates directly to clients, reducing latencies when domain names change, particularly automated DNS topologies for load balancing, failover, etc. Reducing latencies, that is, if you're a paid CloudFlare customer.
I agree they shouldn't take away the "god-mode" /etc/hosts, which is only ever populated very intentionally by sysadmins and power users. If anything, that should be a flag just like the various modes of using TRR.
And finally - it's an open protocol in development, and anyone can set it up who wants to. If you don't want to use Google or Cloudflare, you don't have to. And FWIW millions of people are already using 8.8.8.8 and 1.1.1.1 and Cisco's OpenDNS as their primary resolver. That GOOG and CF are at the forefront of another increment of Internet standards should not be surprising.
Moreover, whatever promises CloudFlare and others make, they're still centralization points and therefore ripe targets for infiltration and exfiltration.
The moment browsers actually solve the privacy problem will be the moment 9/10ths of the internet goes poof and disappears along with their monetization strategies. As long as Google is viable then these are just tricks arguably doing more harm (increasing reliance on centralized vendors, increasing complexity of the software stack) than good.
"0 - Off (default). use standard native resolving"
...
"5 - Explicitly off. Also off, but selected off by choice and not default."
It seems that the plan for the "0 - default" is to switch the users to other modes without the user knowing it, and to keep the behavior off the user must specifically change the option to "5."
or...
It could be prepared for when the user gets asked what they want and then Firefox can remember an explicit "no" as compared to not selection ever made.
/ Daniel (author of the blog post)
Daniel’s own decisions aren’t in question here. If he works for Mozilla he is not more powerful than the whole company.
Having “off” and “off when selected by user” but not other variants still points to the intention of the default state not remaining off. Which is not problematic. Problematic is however naming the state that will obviously be changed “off.”
That's arguably rude, but then your follow-up is exactly the type of conspiratorial bullshit he's implicitly accusing you of, so it was just foresight after all I guess.
If this passes, it's going to be a nightmare to system administrators. Basically, each and every split horizon will be broken.
But, promoting DNS-over-HTTPS in the browser and providing an easy-to-install, separate tool for Windows/OSX to solve through DNS-over-HTTPS is something I could get behind.
Like how on my network I do use dnscrypt-proxy, so everything is already using DNS-over-HTTPS.
It's disappointing to see this coming from someone so steeped in Internet contributions and history and believing it to be a good idea.
Bypassing system DNS is not just an enterprise no-go, it probably will end up reducing privacy.
The idea that ISPs sniff DNS is largely a red-herring, and further it's already easily addressed by DNSCrypt or DNS over (d)TLS.
Furthermore, ISPs in some countries like India forces all port 53 traffic to their own censored servers. DNS over TLS won’t solve that.
I can see app developers wanting this, but as a user, I really hope, this doesn't happen. It's bad enough that many applications today manage their own certificate stores, making the next part of internet infrastructure app-specific seems to me a way to more fragmentation and less understanding or oversight I'd have over my own system.
One reason to desire separate trust stores is that your model of trust is almost certainly not "I wish my application trusted exactly the same CAs as [say] the Firefox web browser and I will incorporate all the same special rules and exception as that browser".
Example: Back in 2016 the US government expressed interest in operating its own public CA. This probably won't happen under Trump of course, but in Firefox this would have been no problem if they met its other criteria, it's easily able to accept a new CA and apply constraints to it. So that in Firefox a US Federal Web PKI cert for whitehouse.gov works, but one for gov.uk or google.com does not. But does your application have that logic? Or would it just blindly trust the new CA because Mozilla added it to their trust store?
The other reason is that your application is (especially if you aren't on the ball enough to run your own trust store) probably not able to keep up with the security treadmill, and falling off may be painful.
Example: If your system depended upon SHA-1 certificates to function, but you used the Web PKI CAs from somewhere like Mozilla's NSS store, magically in 2016 no more new certificates were available. No problem for the browser vendors, they had voted for exactly this outcome. Too bad for your application.
I also noticed that when I configure my Android's proxy settings to point at a Privoxy container that routes through a VPN, I still get DNS-hijacked to my provider's "thepiratebay.org has been blocked for you" page -- this only happens in Chrome mobile, not Firefox mobile. I was used to DNS resolving through the proxy server.
[1] https://github.com/curl/curl/wiki/DNS-over-HTTPS#publicly-av...
IMO, employees should be using whatever their organization has configured for them -- not whatever they wish.
A "regular user" changing the DNS servers on his work PC (joined to an Active Directory domain) to 8.8.8.8, for example, WILL run into problems at some point. They say they have workarounds for this but I think it's safe to say that bugs will likely be found. In the meantime, things will be broken for those users.
Moving this setting into the web browser (where it likely CAN be changed by a user) is not the proper solution.
Can we please go easy on the newspeak? Centralizing resolving to a handful of actors will not improve privacy for the most part of end users.
Besides, I don't see how adding an option for DoH to Firefox is centralizing anything, you're free to set the DoH URL to whatever you like, and you're free to run your own DoH resolver, just like you're free to run your own vanilla DNS resolver.
AUIU, this is currently disabled by default but will be enabled by default in the future.
When that switch is flipped, that's when the "centralizing" begins.
If this were to be disabled by default and forever remain that way, I would be perfectly fine with it.
While I'm sure it's true that Firefox's implementation does this (presumably it provides a setting in the UI whereas it didn't before), there was nothing preventing this with regular DNS. It isn't something that DNS-over-HTTPS makes possible.
Specifying your own DNS server instead of using the system default is a feature that the nslookup command has offered for like 25 or 30 years, so I'm sure a web browser could offer it too.
I thought this might be a good way to ensure that this behavior does not get turned on without my knowledge (whenever the time comes that Mozilla enables it by default).
Unfortunately, browsing broke immediately. I could not access any sites that I tried to access ("Hmm. We’re having trouble finding that site.") until I changed this setting back to the default ("0").
[0]: "Explicitly off. Also off, but selected off by choice and not default."
The difficulty there, I believe, is that /etc/hosts is commonly parsed by your libc, so firefox will have to reimplement /etc/host parsing on top of their libc if it does not expose /etc/host entries in some way (glibc and muslc both don't to my knowledge)
Also will the end goal be to have a new DHCP option so I can point my clients to my internal DOH server?
One of the nice things about this is I can make lookups authenticate. I hope the extend the update protocol into the HTTP context so we have a simple API for updating zones rather than nsupdate.
Does it still account for /etc/hosts blocks? I use a few thousand entries to black hole malware and ad networks and I’d rather not lose this.
1. fetch page of html, e.g., hn front page
curl https://news.ycombinator.com > 1.htm
2. extract urls from 1.htm yyt < 1.htm > 1.txt
(example scanner "yyt" provided below as t.l)3. convert urls to hostnames
g=1.txt k 1
(example script provided below as "1.k")4. retrieve json dns data from doh dns server, efficiently, over a single connection
see https://news.ycombinator.com/item?id=17228745
5. convert json dns data to csv see https://news.ycombinator.com/item?id=17228473
6. import csv into database e.g. sqlite3, kdb+, export to /etc/hosts, export to zonefile for localhost auth dns server, etc.now, when user is reading hn front page, no dns lookups are needed. user already has the dns data. there is no network usage for dns requests, increasing hn front page browsing speed for the user. there are no piecemeal dns requests sent, increasing user privacy.
7. track ip address changes over time, compare answers from different caches, etc. retrieve type 2 (NS) instead of type 1 (A) records, then compare to NS records provided in public zonefiles from icann, public internet scans, etc.
cat t.l
#define p printf("%s\n",yytext);
%%
\200|\201|\204|\223|\224|\230|\231|\234|\235
http:\/\/[^ \n\r<>"#'|]* p;
https:\/\/[^ \n\r<>"#'|]* p;
ftp:\/\/[^ \n\r<>"#'|]* p;
.|\n
%%
int main(){ yylex();}
int yywrap(){}
/* compile with something like:
flex -Crfa -8 -i t.l
cc -pipe lex.yy.c -static -o yyt
*/
cat 1.k /k3 (novice level)
/usage: g=f k 1 where f is list of urls
h0:_getenv "g";h1:0:h0; h1:{:[(#h1[x] _ss "://")>0;h1[x];_exit 1]}'!#h1;h1:{*((h1[x] _ss "://[^/]")+3) _ h1[x]}'!#h1;h2:{h1[x] _ss "[^a-z^A-Z^0-9^.^-]"};h3:{*h2[x]};h1:{h3[x]#h1[x]}'!#h1;h1:?:/h1;h0 0:h1;
\\ / novice alert. this is probably 3x larger than necessary
h0:_getenv "g";h1:0:h0;h1:{:[(#h1[x] _ss "://")>0;h1[x];_exit 1]}'!#h1;h1:{*((h1[x] _ss "://[^/]")+3) _ h1[x]}'!#h1;h2:{h1[x] _ss "[^a-z^A-Z^0-9^.^-]"};h3:{*h2[x]};h1:{h3[x]#h1[x]}'!#h1;h1:?:/h1;if[0=#h1[0];h1:h1 _di 0];h0 0:h1;
\\ ftp -4o 1.htm $1;
fetch -4o 1.htm $1;
wget -4o 1.htm $1;
curl -4o 1.htm $1;
exec k 1d \
|exec tcs cloudflare-dns.com cloudflare-dns.com >1.json;
yyf < 1.json >1.csv;
#t=list exec k j;
#t=hosts exec k j;
t=db exec k j >1.csv;
#t=zone exec k j;
\\
k).Q.fs[{`t insert +:`ts`ip`hn!("ZSS";",")0:x}]`:1.csv
k)t
\\
/ 1d.k
/ urls, hosts
a:0:"1.htm";
a:,/$a;
a:_ssr[a;"\42";""];
a:_ssr[a;"http://";"https://"];
a:_ssr[a;"src=//";"https://"];
b:a _ss "https://";
a:b _ a;
b:{a[x] _ss "[^a-z^A-Z^0-9^.^-]"}'!#a / fail: https://example.com-
c:{1#(3_ b[x])}'!#b;
b:{c[x]#a[x]}'!#c;
b:?:/b;
b:{8_ b[x] }'!#b;
/ http
a:"GET /dns-query?ct=application/dns-json&name=";
c:"&type=1 HTTP/1.1\r\nHost: cloudflare-dns.com\r\nConnection: ";
d:"keep-alive";
e:"close";
f:"\r\n\r\n";
g:(#b)-1;
h:{a,b[x],c,d,f}'!g;
i:a,b[g],c,e,f;
`0:,/$h,i;
\\
int main(int argc, char **argv){
char *b[17];
b[0]="/usr/bin/openssl";
b[1]="s_client";
b[2]="-tls1_2";
b[3]="-no_ssl2";
b[4]="-no_ssl3";
b[5]="-ign_eof";
b[6]="-no_ticket";
b[7]="-tlsextdebug";
b[8]="-servername";
b[9]=argv[2];
b[10]="-verify";
b[11]="9";
/* -host, -port removed from manual */
/* but still found in s_client.c */
b[12]="-host";
b[13]=argv[1];
b[14]="-port";
b[15]="443";
b[16]=(void *)0;
execve("/usr/bin/openssl",b,(void *)0);
}
/j.k
j:("SSSSSS";",")0:"1.csv";
k:_getenv "t";
if[k _sm "list";`0:{,/$j[4;x]}'!#j[5]];
if[k _sm "hosts";`0:{,/$j[5;x]," ",j[4;x]}'!#j[5]];
if[k _sm "db";`0:{,/$j[2;x],".",j[1;x],".",j[0;x],"T",j[3;x],",",j[5;x],",",j[4;x]}'!#j[2]];
if[k _sm "zone";`0:{,/$".",j[4;x],"\n&",j[4;x],".:127.0.0.1:5\n=",j[4;x],".:",j[5;x],":5"}'!#j[2]];
\\
\\
/* f.l */
#define echo ECHO
#define jmp BEGIN
#define p printf
#define nl p("\n")
#define s p(",")
%s xa xb xc xd xx xy xz
xa "\"Question\":[{\"name\": \""
ya ", \"type\": 1}],"
x1 "\"type\": 1"
x0 "\"type\": "[^1]","
yb "\"data\": \""
xw "Date: "
xx \"\},\{\"
xy \"\}\]\}
xz "Sun, "|"Mon, "|"Tues, "|"Wed, "|"Thu, "|"Fri, "|"Sat, "
%%
{xw} jmp xz;
<xz>"GMT" jmp xy;
<xz>{xz}
<xz>" Jan " p(",01,");
<xz>" Feb " p(",02,");
<xz>" Mar " p(",03,");
<xz>" Apr " p(",04,");
<xz>" May " p(",05,");
<xz>" Jun " p(",06,");
<xz>" Jul " p(",07,");
<xz>" Aug " p(",08,");
<xz>" Sep " p(",09,");
<xz>" Oct " p(",10,");
<xz>" Nov " p(",11,");
<xz>" Dec " p(",12,");
<xz>\40 s;
<xz>. echo;
<xy>{xa} jmp xa;
<xa>{ya} s;jmp xb;
<xa>\.\"
<xa>. echo;
<xb>{x0} jmp xx;
<xx>{xy} nl;jmp 0;
<xx>{xx} jmp xb;
<xb>{x1} jmp xc;
<xb>.
<xc>{yb} jmp xd;
<xc>.
<xd>\"\} nl;jmp 0;
<xd>. echo;
\n
.
%%
int main(){ yylex();}
int yywrap(){} /* f.l */
#define echo ECHO
#define jmp BEGIN
#define p printf
#define nl p("\n")
#define s p(",")
%s xz xy xa xb xx xc xd
xw "Date: "
xz "Sun, "|"Mon, "|"Tues, "|"Wed, "|"Thu, "|"Fri, "|"Sat, "
xa "\"Question\":[{\"name\": \""
ya ", \"type\": 1}],"
za ", \"type\": 1}]}"
x0 "\"type\": "[^1]","
xy \"\}\]\}
xx \"\},\{\"
x1 "\"type\": 1"
yb "\"data\": \""
%%
{xw} jmp xz;
<xz>"GMT" jmp xy;
<xz>{xz}
<xz>" Jan " p(",01,");
<xz>" Feb " p(",02,");
<xz>" Mar " p(",03,");
<xz>" Apr " p(",04,");
<xz>" May " p(",05,");
<xz>" Jun " p(",06,");
<xz>" Jul " p(",07,");
<xz>" Aug " p(",08,");
<xz>" Sep " p(",09,");
<xz>" Oct " p(",10,");
<xz>" Nov " p(",11,");
<xz>" Dec " p(",12,");
<xz>\40 s;
<xz>. echo;
<xy>{xa} jmp xa;
<xa>{ya} s;jmp xb;
<xa>{za} s;nl;jmp 0;
<xa>\.\"
<xa>. echo;
<xb>{x0} jmp xx;
<xx>{xy} nl;jmp 0;
<xx>{xx} jmp xb;
<xb>{x1} jmp xc;
<xb>.
<xc>{yb} jmp xd;
<xc>.
<xd>\"\} nl;jmp 0;
<xd>. echo;
\n
.
%%
int main(){ yylex();}
int yywrap(){}[edit]: plus isn't Stateless TLS Session Resumption effectively a cookie?
Even without tickets, having TCP sessions means that server operators can link multiple queries to a single device, even if multiple devices share the same external IP.
This gives server operators more data than plain DNS.
The DNSCrypt protocol can use a unique key for every query in order to prevent this. Since it doesn't use sessions, and all queries are independent, there is no latency overhead.
For DoH, dnscrypt-proxy has an option to disable TLS session resumption. But it introduces some overhead every time a reconnection is necessary.
Although I am not too comfortable with everything moving to HTTP. HTTP 2 was already complex enough, it seems we want to move everything into HTTP, everything away from TCP to UDP. What happen to QUIC anyway ?
Does anyone know how TLS over TCP can be faster than UDP?
But yes, for the common case the performance will be the same.