Hacker News Reader

Comment by deathanatos | Hacker News Reader

> When I remind HN readers that most site addresses are more static than dynamic, I am basing that statement on evidence i have collected.

Sure. But without seeing the other sides argument, I have to wonder if their point wasn't that they're not designed to be stable for the purpose of identifying a service/thing on the Internet; things can and do move and change. Hardware failure is a good example of that. Just like a house address, those too are normally stable but people can & do move. Just with software, it's like we look our friend up in the white pages¹ prior to every visit, which one might not do in real life.

¹oh God I'm dating myself here.

View on HN

That was not the other side's "point". I routinely make the statement: Most sites submitted to HN have realtively static IP addresses, i.e., these addresses can change, but in fact they change only infrequently, if at all.^1 This is not an opinion. It is not a mindless regurgitation of something I read somewhere. I am looking at the data I have, not theorising. From where I sit, there is nothing to argue about.

1. Why do I state that. Because I kept reading about why DNS was created and always encountered the same parroted explanation, year after year. Something along the lines that IP addresses were constantly in flux. That may have been true when DNS was created and the www was young. But was it true today. I wanted to find out. I did experiments. I found I could use the same DNS data day after day, week after week, month after month, year after year.

Why would I care. Because by eliminating remote DNS lookups I was able to speed up the time it takes me to retrieve data from the www.^2 Instead of making the assumption that every site is going to switch IP addresses every second, minute, day or week, I assume that only a few will do that and most will not. I want to know about those sites that are changing their IP address. I want to know the reasons. When a site changes its IP address, I am alerted, as you see with today's change to HN's address. Whereas when people assume every site is frequently changing its IP address, they perform unnecesary DNS lookups for the majority of sites. That wastes time among other things. And, it seems, people are unaware when sites change addresses.

2. Another benefit for me is that when some remote DNS service does down (this has happened several times), I can still use the www without interruption. I already have the DNS data I need. Meanwhile the self-proclaimed "experts" go into panic mode.

I call BS on your second point.

Just run a DNS server locally configured to serve stale records if upstream is unavailable.

As for your first point, the same local DNS server would also provide you with lower/no latency.

This is exactly the sort of comment to which I am referring. Maybe I am just getting trolled. I should just ignore this gibberish. How can something be "BS" if it works.^2 I am using this every day.

I used to serve DNS data over a localhost authoritative server. Now I store most DNS data in a localhost forward proxy.

If "upstream" means third party DNS service to resolve names piecemeal while accessing the www, I do not do that.^1

1. I do utilise third party DoH providers for bulk DNS data retrieval. Might as well, because DoH allows for HTTP/1.1 pipelining. I get DNS data from a variety of sources, rather than only one.

2. If it were "BS" then that would imply I am trying to mislead or deceive. The reverse is true. I kept reading sources of information about the internet that were meant to have me believe that most DNS RRs are constantly changing. I gathered DNS data. The data suggested those sources, whether intentionally or not, could be misleading and deceptive. Most DNS RRs did not change. BS could even mean that I am lying. But if I were lying and the DNS RRs for the sites I access were constantly changing, then the system I devised for using stored DNS data would not work. That is false. It works. I have been using it for years.

> How can something be "BS" if it works.

Nobody claimed it didn't work. The claim that is disputed is it is meaningfuly faster.

> I used to serve DNS data over a localhost authoritative server. Now I store most DNS data in a localhost forward proxy.

I run my own authoritative DNS on my router (tho not localhost. interesting), and have for a long time (since I started traffic shaping to push the ACKs to the front). Like you, I've also enjoyed having superior performance over those using public servers. Everyone says "but you can use 8.8.8.8 or 1.1.1.1! they're fast!." and I (we?) smile and nod.

Just did a quick little test for this comment. Resolving with 8.8.8.8 is fast! And... also between 800% and 2500% slower than using my (and your) setup. high five

Also, the haters don't know something that we do, which is that... sometimes 8.8.8.8 doesn't work!!!

A few weeks ago there was a website I couldn't access from a computer using 8.8.8.8. I thought, "that's odd", used dig, and it didn't resolve. From the same network I tried a different resolver -- worked. Tried 8.8.8.8 again -- fail. sshed a few hundred miles away to check 8.8.8.8 again -- working. tcpdump on the router, watched 8.8.8.8 fail to resolve in front of my eyes. About 4 minutes later, back to normal. "yes, sometimes the internet so-called gods fail."

I'm quite curious why you changed from an full authoritative setup to a proxying one. I've skimmed a handful of your past posts and agreed entirely, so we're both "right", or both wrong/broken-brained in the same way. ;-)

Is there something I could be doing to improve my already fantastic setup?

Using a forward proxy and mapped addresses instead of doing DNS lookups is just a phase in a long series of steps to eliminate the use of third party DNS service, i.e., shared caches,^1 then eliminate unnecessary DNS queries,^2 and finally eliminate the use of DNS altogther. However there are other reasons I use the proxy, namely control over TLS and HTTP.

1. This goes back to 2008 and "DNS cache poisoning". Easiest way to avoid it was to not use shared caches.

2. I created a fat stub resolver^3 that stored all the addresses for TLD nameservers, i.e., what is in root.zone,^4 instead the binary. This reduces the number of queries for any lookup by one. I then used this program to resolve names without using recursion, i.e., using only authoritative servers and RD bit unset. Then I discovered patterns in the different permutations of lookups to resolve names, i.e., common DNS (mis)configurations. I found I could "brute force" lookups by trying the fastest permutations or most common ones first. I could beat the speed of a cache for names not already in the cache. I could beat the speed of 8.8.8.8 or a local cache for names not already in the cache.

3. Fat for the time. It is tiny compared to today's Go and Rust binaries.

4. Changes to root.zone were rare. Changes are probably more common today what with all the gTLDs but generally will always be relatively infrequent. Classic example of DNS data that is more static than dynamic.

"2. Another benefit for me is that when some remote DNS service does down (this has happened several times), I can still use the www without interruption. I already have the DNS data I need. Meanwhile the self-proclaimed "experts" go into panic mode."

Above are the specific claims that were called "BS". One has to do with enabling me to use the www without interruption if DNS stops working.^1 The other has to do with "experts" going into panic mode.^2

Neither claim relates to something being "meaningfully faster."

1. Because I use stored DNS data.

2. Because none of them advise anyone to store DNS data, let alone use it. They opt to promote and support a system that relies on DNS to work 100% of the time.

I think there's a case that's missed here though: IPs change when you move as well. There's a number of services which have different IPs depending on whether i use my home connection, or mobile, or go to the office in another city. It's not that they will stop working, but they'll be slightly less optimal.

> they perform unnecesary DNS lookups for the majority of sites

Is it actually unnecessary if the IPs can change? I'm fine with the extra 20ms on the access every once in a while in exchange for no mysterious failure every few years.

In other words, DNS load balancing or something similar.

I am not really I fan because I like to choose the IP address, instead of letting someone else decide. I believe in user choice.

In some cases I have found the "most optimal" IP address for me is not always the one advertised based on the location of the computer sending the query.

It is like choosing a mirror when downloading open source software. I know which mirrors I prefer. The best ones for me are not necessarily always the ones closest geographically.

As for the question, the answer is yes. Because if it did not change then the query was not needed. If it does change then I will know and I will get the new address. The small amount of time it takes to get the new address and update a textfile is acceptable to me. I may also investigate why the address changed. Why did this HN submission go to the front page, why does it have so many points and comments. Some people are interested when stuff happens. I actually like "mysterious failures" because I want to know more about the sites I visit. Whereas an extra delay every time a TTL expires, for every name, again and again, over and over, every day, that is a lot of time cumulatively. Not to mention then I have to contend with issues of DNS privacy and security. When I started weaning myself off DNS lookups, there was no zone signing and encrypted queries.

The approach I take is not for everybody. I make HTTP requests outside the browser and I read HTML with a text-only browser. I do what works best for me.

> I am not really I fan because I like to choose the IP address, instead of letting someone else decide. I believe in user choice.

Do you also object to anycast?

Why would I object. If it works, I will use it.

For example, I ping 198.41.0.4. I choose to ping that address over all the others, e.g., www.google.com or whatever other people use. That is what I mean by user choice. I know the address is anycasted. Where the packets actually go is not something I get to choose. It would be neat to be able control that, e.g., if source routing actually worked on today's internet. But I have no such expectations.

How do Tor users know that an exit node IP address listed for a foreign country is not anycasted and the server is actually located somewhere else.

Maybe check against a list of anycast prefixes.

http://raw.githubusercontent.com/bgptools/anycast-prefixes/m...

https://raw.githubusercontent.com/netravnen/well-known-anyca...

https://www.ietf.org/archive/id/draft-wilhelm-grow-anycast-c...

Interesting! That runs in direct conflict to what I learned eons ago (pre-web) for "why DNS?". (Or maybe, it conflicts with what my faulty meat brain remembers.)

The gist was "we have DNS because without it, people would have use numbers. people don't like numbers." DNS is primarily there to provide semantic meaning". The fact that it allows the numbers to change is.. a secondary bonus.

DNS exists for the same reason as variable names instead "variable numbers" (like a, b, c, d, &c) For us humans to provide semantic labels to things.

(an aside, "variable number" is exactly how things are still done in math and physics. This amuses me greatly.)

I did not study computer science so anything I know I learned from reading what was available in textbooks and on the internet itself. I learned DNS exists because the number of hosts was growing too quickly to keep updating a HOSTS file. The HOSTS file permits me to assign semantic meaning, i.e., names, to IP numbers. I can name hosts however I choose, and in practice I still do, because I like very short names. A simple analogy perhaps might be assigning names, images, sounds, etc. to different stored contact numbers on a mobile phone. The owner of the phone can control the semantic meaning assigned to the number, rather than delegating all control over this to someone else.

DNS, as I see it, lets someone else assign the names, i.e., the semantic meaning. Thus, assuming I am an internet user in the pre-DNS era, with the advent of DNS, I do not have to keep updating a HOSTS file when new hosts come online or change their address. This reduces administrative burden. The semantic meaning was already controllable pre-DNS, via the HOSTS file.

Many times I have read the criticisms of IP addresses as justifications for DNS. For example, IP addresses are (a) difficult to type or (b) difficult to remember. I simply cannot agree with such criticisms. As time goes on, and the www gets continually more nonsensically abstracted, I like IP addresses more and more.

> The gist was "we have DNS because without it, people would have use numbers. people don't like numbers." DNS is primarily there to provide semantic meaning". The fact that it allows the numbers to change is.. a secondary bonus.

This is before i was born, but that sounds more like the reason why /etc/hosts was invented, which predates dns.

> an aside, "variable number" is exactly how things are still done in math and physics. This amuses me greatly.

Variable names are usually idiomatic within a field/carry some semantics. e.g. k is angular wavenumber, omega is angular frequency. r is displacement. etc. They just use short names to prevent the name from distracting from the shape of the equations it's used in, so that it's easier to say things like "this behaves like a transport equation but with a source term that's proportional to the strength of the Foo field squared" or whatever.

Lots of phenomena have very similar governing equations, so downplaying the names of variables in favor of the structure/context they're used in allows for efficient transfer of intuition.

> Because I kept reading about why DNS was created and always encountered the same parroted explanation, year after year. Something along the lines that IP addresses were constantly in flux. That may have been true when DNS was created and the www was young. But was it true today. I wanted to find out. I did experiments. I found I could use the same DNS data day after day, week after week, month after month, year after year.

I have never in my life heard anyone claim this as the reason for dns.

The usual reason given is two fold:

Flat /etc/hosts files were getting large enough to be annoying.

The set of all dns records as a whole change constantly. Individual records dont change very much. But the time between at least one record changing is very small.

Both of these things are even more true today then they were when dns was invented.

Aside: How did you get the superscript? Is that supported by HN's formatter or is that just a literal superscript character?

Unicode has code points for superscript/subscript digits. That one is U+00B9: https://www.compart.com/en/unicode/U+00B9 (So it's "normal text", as far as HN is concerned. Note that HN does filter some things, like emoji.)

I was on macOS when I typed it, there it's Control+Cmd+Space, and then search for "super" which gets close enough.

On my Linux machine, I can either do Compose, ^, 1, or Super+e and then search for it. (But both of these require configuration; either setting a key to be Compose (I sacrifice RAlt), or setting up whatever it is the IME I have is for Super+e.)

Not only super/subscript, there's also some convenient fractions: ½ ⅔ ⅜, etc

Reply on news.ycombinator.com