I've taken the opportunity to write up the story of the first time Varnish Cache's 503 was heard around the world, or at least in Norway:
http://varnish-cache.org/docs/trunk/phk/503aroundtheworld.ht...
2,337 karma · joined June 15, 2010
http://varnish-cache.org/docs/trunk/phk/503aroundtheworld.ht...
Max has many good points about error messages in general, but all of them require access to out of band information.
We in the Varnish Cache Project do not have access to that information, we dont know who runs the varnish instance or what kind of information they serve to what kind of clients.
This is why the default '503 message only exposes the "XID" nonce: That allows the administrator of this cache instance to find all the details in the log files.
Varnish Cache users who want to present something else can do that from VCL, and I'm pretty sure Fastly normally does.
But when all else fails, and here it must have, Varnish Cache errs on the side of caution.
Back then /everything/ was a special case and everything needed to be programmed as such.
This is literally why JCL is a nightmare.
The genius of Ken & Dennis was to boil all the special cases down to one simple abstraction.
(Look at the socket API to if you want an example what happens when people dont understand the importance of sticking with the dominant abstraction.)
But it is one of those things were I just chose to trust the OS to DTRT, and automatically reaped the benefits of advances "under the hood".
As I said above, it is not obvious to me that we would gain much over what we already do, but I am keeping an eye on it.
By this people mean things like printing the wrong URL in a full-page sunday newspaper add or splitting traffic based on first digit of customer-number and stuff like that.
Some of the screwups Varnish-ops have fixed are truly the stuff of legends, but telling them belongs to their heros :-)
Back then you couldn't buy machines with 1TB of RAM, most systems were 32bit and limited to about 3GB usable RAM.
Varnish was written with 64bit in mind from the start, but it also had to work on 32 bit systems, so we had to be able to use disk-backed VM to get past a couple of GB.
If you have a lot of programmers churning out code, languages like golang makes it harder for them to make mistakes, sometimes at the cost of performance, but 99.9% of the time, performance does not matter.
If performance matters, there is no better language than C to bend the hardware to your precise will, but that comes at a cost of programming difficulty.
Unless you are in numerical HPC or kernel programming, you do not need to think about NUMA, but it is not black/white, it is a slope.
Varnish tries to do certain things "NUMA-friendly", without actually being "NUMA-aware" as such, and the kernels seem to do a really good job with that.
Redesigning that code has not bubbled up to the top of my TODO list yet, and shows now signs of doing so.
But if you need persistence, there is a commercial version of Varnish which has it, but I have not been involved in that.
There still is a cost differential between 1TB RAM and 1TB disk, and that matters in the real world.
Varnish lets you decide where you want to spend your money, if you want to run straight out of RAM, it can do that, if you want to have a bigger cheaper and slightly slower cache on disk or ssd, you can do that too.
But the most important question is your data set and your traffic pattern, and whatever that is, Varnish can deal with it, given suitable hardware.
NUMA is interesting, I've had a lot of fun with it on a system to control the adaptive mirrors in ESO's ELT telescope, and it is /amazing/ what you can do with a modern server, even on a vanilla kernel.
However, it is a lot harder to exploit NUMA in an application like Varnish, because the requests arrive as a randomized stream, and the effort to figure out which NUMA pool is best situated to handle the request, is not significantly different from just handling the request to begin with.
We used to serve a cache hit in seven system calls, an accept(2), a read(2), a write(2) and four timestamps, and as a result, Varnish servers ran with CPU's >70% idle.
If we had added NUMA-awareness on top of that, we would on average Nnuma-1/Nnuma of the time add at two system-calls to migrate the request to another NUMA domain, and that made little sense.
Since then NICs have become NUMA aware and since we use scatter/gather I/O, that matters a lot more than which CPU core did the overhead processing.
Where NUMA may become mandatory is 100G and 400G networking, but very few people seem interested in running that much traffic through a single server, for reasons of reliability, but I'm keeping an eye on it.
https://trends.builtwith.com/Web-Server/Varnish
My personal estimate is that around 20% of all HTTP traffic pass through a Varnish instance somewhere, many of which are not public-facing.
As to why people use Vanish, I think the main reason is that VCL gives you full and instant control over your HTTP traffic, and people really like that, because it lets unify heterogenous sites and pamper legacy systems as necessary.
The reason why Varnish does not have built in TLS, is a matter of security design and flexibility.
First, as to flexibility:
If we added TLS support to Varnish, we would have to pick a TLS library, which means our users would automatically be locked into that library.
This would violate one of our core principles, which is "Tools, not policies".
And given the state of TLS implementations, I /really/ dont want to make that choice, I want to leave that to the person responsible for the security at the site running Varnish.
Second, I think it is sound security engineering to confine the server certificate in a process which does that one thing and does it well.
I know several sites that have two different TLS frontends in front of Varnish, using different TLS implementations, so that whenever a CVE hits one of them, they can just shut that half down until the issue is fixed, and still remain in production. That would not be possible if TLS was bolted into Varnish.
Does anybody know the situation in any EU countries ?
The claimed "time ~ experience" correlation certainly exists, and in a long established discipline, like plumbing, experience is almost unquestionably a virtue.
In a young discipline, such as computers, there is a very concrete risk of confusing experience with not-moving-with-the-times, certainly on the party claiming experience.
So it is probably very much a case-by-case judgement call ?
But I did not think then, and still do not think now, that they are core material in bread&butter CS algorithm courses, I consider them more of a research-curiosity, so I cant say I even thought about bringing them into the article.
My main objection was that some of the algorithms praised in theoretical CS have near pessimal behaviour on actual hardware, and have had so for a quarter of a century, yet these "details of implementation" were deemed a waste of educational bandwidth.
I've had a fair bit of communication with people in boths ends of CS education about that piece, and I think by and large most agree that I have a valid point there.
But nobody wants "Algoritms %03d" to turn into "Optimizing your code for $CORPS microarchitecture %03d", and modern architectures are horribly complicated, in particular from a performance point of view, so sifting out the most representative and relevant phenomena and finding a way of presenting them in education is not easy.
On CS TA told me that they hand out my article mid-semester, and ask the students to identify algorithms in their textbook which are robust or vulnerable to actual hardware performance. I'm torn between being flattered and thinking it is a bit of a cop-out.
As far as cache-oblivious algorithms go: I'll consider them when they prove they are worth their often formidable complexity.
Yes, it would be nice to never have to think about about page-sizes or cache-line widths and all that again, but if only people with a phd in those exact algorithms (many of which are patented) can debug programs which use them, the cost/benefit tilts.
I have spent almost four decades becoming better at writing C and I write much better programs in C than I do in any other language because of it, and if you asked me to write something very important today, I would trust the result more, if I wrote it in C, even if that were, on some objective scale, not the best language for the task.
With that said, a lot of my recent projects, I have chosen python3, which I am now, after some years of "toy-projects", starting to feel quite comfortable with.
But given that any slight, no matter how insignificant, no matter against which part of the Venn diagram it was directed, invariably gets labeled "antisemitic", I think people like me can be excused if we express ourselves with less exacting clarity, than the people in the Venn diagram demands.
If the people in the Venn diagram think the distinctions are very important, a good place to start is to use very distinct colors.
That's why countries have spent centuries refining how to properly document and dispose of cases of death.
But somebody needs to know where to send the official certificate of death to in the first place.
In my childhood, the death-notices in the local newspaper did the job, that's no longer the case.
https://queue.acm.org/listing.cfm?sort=publication_date&orde...
If you are anything like me, you probably attended less than a handful of funerals until your fourties, and then, as part of growing "really" up, making sure the dark respectful clothes are always ready became a thing.
The problem is we built the internet before we turned 40, so we never thought about how deaths figured into it.
I still think the two most important aspects are A) Ensure access. (ie: write down crucial passwords) and B) Making sure the news gets round.
Failing A) may leave practical and economical heart-burn.
Failing B) will make the people you leave behind much more uncomfortable and sad than you would want.
And, of course: Make sure to tell people how much you appreciate them, while both you and they still are.
Feel free to ask any questions...