SSL revisited | Hacker News Reader

SSL revisited

varnish-cache.org

59 pointsmro67 comments

I'm not sure how I feel about the Political PostScript section. It raises the following points against the concept of "SSL Everywhere":

- "you don't want to bog down your countrys civil defence agency with SSL/TLS protocol negotiations if their website is being deluged by people trying to survive a natural disaster"

- "there are people who do not have a right to privacy"

- "SSL Everywhere will force institutions to either block any internet connectivity or impose Man-in-The-Middle proxies"

- "SSL Everywhere [gives privacy] to the actors I think deserve it the least"

- "shady behaviour of big transnational, and therefore law-less, companies have been exposed by security researchers (or just interested lay-people) who ran tcpdump"

- "SSL Everywhere puts all traffic in the shade"

--------

My opinion:

Without TLS, the Internet is either read-only or has reasonably high security risk.

I also noticed varnish-cache.org was served to me over TLS.

> SSL Everywhere will force institutions to either block any internet connectivity or impose Man-in-The-Middle proxies

MITM proxies are already how many organizations handle web filtering. (aka "transparent proxies") TLS just necessitates that the organization have some control over the device being MITM'd to do so. (to install their cert)

... And now they also need to circumvent Googles cert-pinning and other attempts to twart even legally mandated MiTM proxies...

Either all MiTM needs to be outlawed (with actual laws) or protocols need to recognize that laws mandate MiTM some places, and accommodate that with minimal loss of security and privacy.

The current weapons race just makes things more and more broken.

Local CAs are allowed to override even cert-pinning for this reason.

Right, so when you're guest at a company you have to install their CERT on your device to use their guest-wlan ?

If you're an employee you have to put the company CERT on your smartphone ?

How does that improve your security ?

Guests shouldn't be on the corporate network anyway. Either companies will provide a guest WiFi signal that is physically separate from the corporate network (that's what my employer does), or they will ask guests to provide their own connectivity via LTE.

If a federal site can survive being DDOSed, it can probably survive normal traffic.

And I fundamentally disagree with the premise that there are people who do not deserve privacy. I can not think of a single person I would wish complete exposure of their lives upon.

Some people do not have the rights to online privacy. For example, when accessing a website from the library of a jail. The authorities want and need to know what you do, this for some obvious and understandable reasons. I agree this is a particular case, but this is still a case for allowing a website (as long as this is anonymous usage of it) to be available without SSL.

Jails can still use MITM monitoring proxies without disabling SSL: they just have to install certs on the machine.

Does it work with sites like Google from Google Chrome where browser knows about their public keys? I think that this will be wide practice in modern browsers.

HSTS/HPKP headers could be stripped by proxy but preloaded public key list probably will require custom browser build.

Yes, it does. Google intentionally adds exceptions from error reporting in the case a root CA was added to the OS.

Even for read-only encryption is important for privacy. An observer can conclude your interests from what you read and can manipulate the contents.

Any security measure either works for everyone, or for no one. There's no way to give good privacy to "good" people and easily compromise it for "bad" people, for any definition of "good" and "bad".

If "bad" people outnumber "good" people in your world view, then yes, it's entirely reasonable to deny everyone privacy.

What are the institutions that are hell-bent on listening to everyone's traffic? China government? OK, serve them non-sensitive content via plain HTTP; they'll filter or alter it to their liking. Possibly you're fine with this. NSA? Well, maybe something can and should be done to it that we can't do to the China government?

(Not to self: when planning a deployment, consider other options before turning to Varnish cache.)

> The most obvious example is that you don't want to bog down your countrys civil defence agency with SSL/TLS protocol negotiations, if their website is being deluged by people trying to survive a natural disaster.

If TLS overhead is the last straw that brings down a web site, the site's infrastructure isn't suitable for delivering safety-critical information.

This. Google measured TLS overhead on their servers – it was very small. If you can do TLS on Google scale, you can do it anywhere.

It really depends on what your bottleneck is. If your bottle neck is already CPU, then TLS is not that big of a deal. If, OTOH, your bottleneck is disk, then implementing TLS can create substantial overhead.

What?

How does TLS increase disk usage?

there's more to it than that I guess.

TLS adds quite a bit of memory overhead, it increases the network bandwidth due to padding, and if your CPU is eqipped with AES extensions (modern ones) then you still do compression usually.

it can be difficult to cache certain things with TLS also- since you can't do transparent caching. (although this is more a problem for those running squid proxies at work on a stretched out line).

TLS is certainly an overhead and it's not required in cases where I'm checking a bulletin about earthquakes.

How so? If your bottleneck is disk, you probably have quite some CPU cycles to spare, because the CPU is waiting for I/O most of the time.

Are you willing to pay increased taxes, so that FEMA (Or your countrys similar) can afford to run 100.000 servers, in order to "do TLS on Google scale", so that they can get emergency orders out for civil defence ?

Have your forgotten (or maybe never seen ?) CNN's traffic graph from 9/11 ?

Experience so far is that emergency services web-pages tend to die the instant they are most needed. Addint TLS will just make that happen even more.

> Are you willing to pay increased taxes, so [...] that they can get emergency orders out for civil defence ?

Unquestionably, yes.

You really need to update your facts. CPU overhead is already less than 1%. https://istlsfastyet.com/

UPDATE/CLARIFICATION: The author DID NOT say people don't deserve privacy, he said "there are people who do not have a right to privacy" I misread that when he really meant that LEGALLY they do not have a right, not that he personally believed this. I would like to apologize for misrepresenting the author's intent.

I'll start by saying I don't agree that some people don't deserve privacy. I just can't get behind that in literally any form. More and more computers and what we store on them are an extension of our minds. By this logic you are more or less reading my thoughts by knowing what I doing online (I could write a whole paper on this).

That said kudos to the author of varnish for actually coming out and explaining why he feels this way. Too often there is no insight into why something was done a certain why (or why it wasn't done at all) and anything that sheds light on that is a good in my book. I understand and agree with where he is coming from in relation to it adding attack surface, complicating the code, and tying him up from working on other things. Those are all legit reasons coupled with the fact that this you can just use something like HAProxy or Pound in front of varnish.

For me nginx caching is good enough for my personal use and at work we are not to the point that using varnish would provide big enough gains to offset the time to implement it (we are SSL-only). That said I will continue to keep my eye on varnish because I have used it before and quite liked it.

Let me just make absolutely clear: It's not my opinion that certain people don't deserve privacy, it is the law of the land, duly enacted and ratified by legitimate governments.

If you want to change that, vote.

Apologies, the line:

> The next big issue is that there are people who do not have a right to privacy. In many countries this includes children, prisoners, stock-traders, flight-controllers, first responders and so on.

and a comment in this thread

> And I fundamentally disagree with the premise that there are people who do not deserve privacy. I can not think of a single person I would wish complete exposure of their lives upon. (https://news.ycombinator.com/item?id=9454402)

caused me to misinterpret that. Do not have a right != Do not deserve. I am very sorry that I implied something that is not true. I am editing my original post to clarify this.

What does that mean then?

> But one of the biggest problem I have with SSL Everywhere is that it gives privacy to the actors I think deserve it the least.

Some deserve privacy more than others? How does that work?

How many mentions do companies get in national constitutions ?

They're not mentioned one single time in the US or the Danish Constitutions, yet, both countries they more or less run government now.

UN's human rights don't mention them either.

Companies have no claim to human rights, like privacy, because they are not humans.

And if you think all humans, no matter what, should have a universal, unabbridgable right to privacy: Vote in your elections, and if nobody is worthy of your vote, get yourself elected instead.

But right now there are fully valid laws that says certain classes of humans do not get privacy, and the government which enacted those laws are legit, elected and have every right to enforce their laws.

I get that at the moment some don't get privacy. I meant the "deserve it the least" part. It seems to contradict "It's not my opinion that certain people don't deserve privacy".

I don't think deserving privacy has a scale. I understand your "deserve least" as "I'd rather they didn't have privacy". Feel free to correct me.

Without discussing any of the political points (which felt out of place), this simply means that we will continue to not use Varnish.

Nginx' caching is good enough that it's not worth the pain of trying to implement some wacky setup just to use Varnish.

It's straightforward to run Pound or HAProxy in front of Varnish to provide SSL-terminatation. I wouldn't consider it a wacky setup. The performance gains over Nginx for static/anonymous caching is significant.

It adds two more layers of software to configure, monitor and maintain, and my own tests disagree on the performance gains for a fairly typical Wordpress install.

The response times were virtually identical, and the load on the box was significantly lower with Nginx over varnish, so the simplicity was the deciding factor.

If you don't need varnish, it would be pretty stupid to start running it...

Varnish has a few bonus features (like caching the gzipped version of responses), which could still make it a useful addition.

We've settled on Nginx for a few of our use cases (to good reception), but there are other use cases we're still evaluating.

It's a good product for what it does -- I've rescued more than one site using it -- but the politics behind it have always been a little weird. Some of it is subtleties of Danish culture and humor not coming through.

But if you're actually running Varnish the way he'd like you to, chances are something isn't quite right with your architecture. Poul-Henning has strong opinions on everything from virtual memory to life on Mars and I really don't need that stuff to be part of my web stack.

A little personal maturity to match the product maturity would help both advance.

I have opinions on life on Mars ?

How is TLS Termination/load balancing > caching proxy > Apache/nginx/whatever "something wrong with your architecture"?

Obviously depends on the "apache/nginx/whatever" side, especially if "nginx" can do everything varnish does in your scenario and moreso when "whatever" includes in-app caching and load balancing strategies.

Varnish grew legs as part of saving the clusterfuck that is WordPress. Custom cache rules up a layer from a broken old code base is AWESOME. Maybe also a red flag.

Refusing to use a piece of software because the author has "strong opinions" seems very strange to me. Most of the best software I use is "opinionated" in some way. The trick is to pick software whose opinions best match your own.

Actually the trick is to pick software that meets the needs of the technical problem at hand.

Getting a read on where the product is headed -- by being aware of corporate motives or by reading developer's ranty blog posts -- can be part of the strategy. Percival, Torvalds, Fried -- I kinda know what to expect from them moving forward.

This guy is playing a similar game but not doing it right.

I agree with what you said about picking the right tool for the job, but I don't see how he's "not doing it right"

I 100% agree with the view that caching and tls termination/load balancing are two different tasks, suited to two different tools.

The stated reason for this approach is keeping the existing excellent solution, from becoming worse without any real gain.

Yes the author has stated views about the use of tls "everywhere" - specifically because varnish doesn't handle tls, those opinions don't affect the tool at all.

Edit: s/told/tls/ damn you autocorrect!

I'm all for reducing complexity but ignoring the bigger problem so you can focus on a smaller one isn't necessarily the path.

Customers are trying to solve a pretty basic, common problem here. I don't see how a ranty, opinionated position paper moves anyone closer to the finish line.

All it did was further influence my opinion of where Varnish would likely be in five years. While I appreciate the candor I'm not sure he did himself any favors.

I have a really hard time following your argumentation, because it seems to have very little to do with both reality and what I wrote.

What I did WRT moving people closer to the finish line was to implement the PROXY protocol, so that using a(ny) preexisting and well-tested SSL-termination solution works seamlessly with Varnish.

IMO, that is a far superior solution to adding a lot of security critical code to Varnish which, at the end of that huge effort, doesn't work any better.

As I wrote in my piece: "the world really don't need another piece of code that does an half-assed job at cryptography"

And doing a full-assed job only makes sense if you have the resources, competence (important with crypto!) and the result makes a positive contribution, one way or another, which offsets the cost of its production.

Nobody has yet been able to point out what the positive contribution would be, compared to a solution where SSL termination is its own layer.

Do you know something about that which I don't ?

If so, please share...

You did a little work, avoided doing a lot more work, and justified it with "more components = better." Fine. Not the only approach but certainly not a radical one either.

You could have stopped there. My feeling is that you should have.

But the rant part of your post? Evoking HeartBleed and "I told you so" and Snowden and digs on BSD and "big transnational, and therefore law-less, companies." Well, that makes you look a little wacky.

By doing this you attracted unnecessary attention to yourself (perhaps the point) but also generated no positive goodwill for the product. Worse, it made me question the motivations behind the technology decision.

It's not a good tech post and it's not a good marketing post.

As I stated, there are people who do this sort of thing well. Your post is an example of not doing it well.

So we have established that your factual knowledge of varnish is severely lacking (and/or skewed) and now you just demonstrated that you have no idea who you are lambasting either.

Let me know if you ever want to have a fact-based discussion, in the meantime, don't get cold up there, on your high horse.

I gave a talk called "NSA Operation Orchestra" some time ago, I recommend you watch it, it might give you something to think about.

See, that's just the thing. I tuned Varnish for 32 bit machines (a terrible idea), I sent in patches. I love everything about OpenBSD. I think the NSA are a bunch of cocksuckers. And I also think that when you combine all those things in a post talking about a feature decision it makes you look like a lunatic. Talk to Percival, he gets it. And thank you for the work on Varnish, it seriously saved several sites.

So what you're saying is that because you disagree with me, I'm a terrible person ?

Needless to say, I don't agree.

> You did a little work, avoided doing a lot more work, and justified it with "more components = better." Fine. Not the only approach but certainly not a radical one either.

Your interpretation of this is ridiculous.

You may as well dismiss Linux because the kernel doesn't include an ANSI SQL compliant database and runtime environment for perl, ruby, python and php.

There is immense value in having several small, specialised tools that can be used together to form a solution.

Likewise there is immense value in having a simpler solution with fewer parts -- when appropriate and when possible. This is part of what we decide as engineers.

It's also easier to sell someone on your component strategy when you don't trash every possible piece that could plug in to the hole you've left -- from operating systems and open source crypto packages to government agencies and transnational corporations. But that little observation seems to be getting lost in the noise here.

Relatively recently I wrote a purpose-built web server that supports HTTPS and links with libssl. The experience was both good and "interesting". The good is that it's actually pretty easy to use libssl, even though the API is pretty terrible. Other than the configuration you already see in products like apache2/nginx/etc. there aren't many knobs and levers to worry about. You simply hand a socket over to libssl and does the heavy lifting. Then you use the libssl functions to read/write data and you can even use select() and friends to check if the socket is ready.

The "interesting" part comes from the fact that a socket may read when you want it to write and vice versa: the underlying protocol is more complex than just pushing bytes. This means that the socket needs to be able to read and write when you want to do one of those operations. There is also fragmentation of your data. You say "send this 8 KB buffer", yet it gets sent in pieces (cipher blocks?), which can lead to some interesting issues (my server was for video streaming, and sending an incomplete frame resulted in artifacts). I solved some of this by enabling SSL_MODE_ENABLE_PARTIAL_WRITE (don't block until everything is sent, just tell me what succeeded).

What I'm trying to say is that not enabling SSL because its code is a mess is an example of "the enemy of good is perfect". While Varnish refuses to add SSL support, nginx has it. That's why I use nginx and not Varnish.

I'm kind of surprised about this repeated "I just use nginx" argument, it doesn't make sense to me at all.

If your site runs fine, in all situations you care about, without FOO, you would be pretty lame if you increased its complexity with FOO nonetheless, for any value of FOO.

That is more or less exactly the central argument of the piece I wrote.

The KISS principle dictate that I do not add SSL/TLS to Varnish, because it would just increase complexity without any comparable net increase in benefits.

Yes, it's probably (slightly) more work to configure a SSL-terminating proxy and varnish, but that is the maximal benefit you can hope to obtain if I implement SSL/TLS.

On the other hand, having SSL termination in a clearly defined separate layer gives you at least the following benefits:

You can change implementation in one layer without affecting the other.

You can scale one layer separate from the other.

You can have different administrator access in one layer than the other.

You can scale the SSL layer for CPU and the Varnish layer for RAM.

You can have multiple independent implementations of your SSL termination, thereby vastly increasing the chances that you don't have to shut down next time some SSL library breaks.

&c &c &c

I for one appreciate the approach taken - the constant drive for all-consuming "one size fits all" tools is ridiculous.

Any eta on v4.1 with PROXY support? I didn't find much when looking quickly (and from a phone)

On the other hand, I don't want China to bog down some contry's civil defence (or any other website) by DDOSing it, injecting JS into unencrypted baidu traffic. Which one is causing more problems right now?

Right now, the biggest problem is the unlimited invasion of privacy done by transnational companies accountable to nobody.

NSA could, in theory, be reigned in by the government, but nobody in the world has the power to stop the transnational companies deconstruction of everybodys indentities.

For instance, I have never agreed to FaceBook's terms and conditions, but they track me, like everybody else.

I can sympathize with the author's feeling. TLS is incredibly complex. It seems that TLS libraries are all bad, setting them up is incredibly difficult, and hell, all the complaints about OpenSSL are not enough to do justice to its quality.

But no, Varnish is the optimal point for doing encryption, and placing anything on its front is contrary to any reason somebody would have to use it.

What makes varnish more optimal than a dedicated tls termination and load balancing layer?

After you add all that, what are you using Varnish for? Cache? Placing your cache at the load balancing instead of fetching pages from the network does actually save processing and memory.

If you use Varnish, it's almost certainly sitting at the best point for doing TLS termination.

This article is written as if "Should we use HTTPS?" was still an open question. That boat has sailed.

If you can't allow SSL on your network, you can't allow use of Google and about 2000 sites which browsers will not even try opening via HTTP:

https://code.google.com/p/chromium/codesearch#chromium/src/n...

Jail libraries and magic-cookie-hunting hackers have an option of installing their own CA certificate. This is supported by all browsers. It's not hard, even Lenovo malware can do it.

"Should we use HTTPS" is very much a closed question, and for a lot of sites the answer is a resounding "NO".

The fact that you might not use those sites doesn't mean that we who deliver tools for them can just ignore them, or even worse, impose our political agenda on them.

You can do HTTPS with Varnish if you want to, but you'll have to do it with the architecturally and security-wise most sensible configuration: With a SSL terminating proxy in front of Varnish.

And again: Talk to your legislators about peoples right to privacy, I'm just pointing out that such laws exist, I'm not writing them (or for that matter agreeing with them.)

In places like jails, schools, and libraries (where the owner can add a compromised CA cert and users don't have permissions to remove it) it's entirely possible to MITM and decrypt all TLS traffic, so I don't get why you're still arguing as if that wasn't the case.

While the long tail of sites isn't on HTTPS yet, the most popular ones are HTTPS-only already, and the result is that in Chrome HTTPS sites are browsed more often than HTTP sites:

https://plus.google.com/+IlyaGrigorik/posts/7VSuQ66qA3C

If you're interested in how you might isolate the private key in a separate process, as proposed by this article, I sketched out an implementation a year ago: https://www.agwa.name/blog/post/protecting_the_openssl_priva...

I hope to update it soon to support ECDSA keys in addition to RSA. (This wasn't previously possible, but I think it's now possible with OpenSSL 1.0.2.)

The case against the trendy push for "HTTPS always, no exceptions" is largely a set of corner cases everyone else is comfortable to ignore, so I'm glad someone is making it.

I have yet to see an opponent of https everywhere address the fact that https provides important authentication. Isn't it important that when you go to your governments emergency status page you see what they published rather than the emergency rick roll that my arp spoofing laptop served you? Are you okay with Comcast injecting ads and verizon adding supercookies to your sessions?

The none cypher provided this without encryption, but it's long been deprecated.

HTTPS doesn't provide authentication. It only communicates authentication, and that is from the untrustworthy and widely trojned CA-concept, which is as broken as it almost can be.

There are other, far better authentication methods for things like emergency services, and I'd rather have unauthenticated information, than no information at all anyway.

You also don't need authentication to stop ISP's being stupid, for that Integrity is all you need.

As bad as the CA system is, I have not seen evidence that it's 'widely trojaned' or broken. The fact that a falsely minted certificate is such big news is evidence to the fact that it is working pretty well despite it's flaws. And it is certainly better than no authentication at all.

Obviously it's not perfect, but being not perfect is no excuse for refusing to use what we've got right now. And it's not a choice of unauthenticated information or no information, it's a choice between authenticated information and possibly wrong information.

Trusting ISPs to have integrity is in my opinion much more absurd than trusting CA's. CA's have a financial motivation to keep their CA status which browsers can revoke. ISPs have nobody keeping them in line.

And you think NSA with their enormous budget and a mandate to collect "everything" looks askance at the CA's and go "Nope!" ?

Really ?

How many of the root-certs that are in your browser by default do you actually trust ?

What objective evidence is there, that any of them can be trusted ?

Nobody says that. We are all aware of that. But we shouldn't make it easier for our local ISPs or WiFi access operators to spy on us. Because those very probably don't have the CAs compromised.

We don't need Varnish anymore, we have Nginx now. Does anyone here use Varnish because Nginx isn't good enough? If so, what is Varnish better at than Nginx, and by how much?

Reply on news.ycombinator.com