Hacker News Reader

Comment by zingplex | Hacker News Reader

Wouldn't implementing TLS in the kernel result in a substantial increase in the exposed attack surface? This seems to me as if it would be similar to the time that Windows moved the graphics into the kernel and the havoc that ensued thereafter.

View on HN

It appears the design of KTLS is intended to minimize the complexity necessary inside the kernel; the TLS handshake is handled in user space and then the TLS state is transcribed to a KTLS file descriptor which handles the traffic thereafter. This means the very complex public key part of TLS is user space. The kernel isn't involved with certificates etc. The kernel handles the symmetric cryptography once the user space handshake is complete. So yes, the net attack surface inside the kernel increases, but it isn't as though they are embedding some OpenSSL equivalent into the kernel.

The paper reports a relatively small performance improvements. But I suppose if you're Facebook (they and RedHat implemented this) and performance improvements are measured in millions of dollars you care a lot about 7% less CPU utilization.

The performance gain is in latency consistency, it seems [0] though this chart is silly, since I'm guessing the X axis is time, and you can't chart a centile, that's ridiculous.. Anyway, even the throughput improvement could be worth it, but I think the main improvement is in latency.

https://twitter.com/tgraf__/status/904475786622640128

You can chart a centile. Each point can be the centile of X measures.

> you can't chart a centile,

Why? If you have servers with millions of qps, the graph would show the 99% latency over, say, a second or ten.

I think this graph is pretty standard? But I'm happy for suggestions of better ones.

There are constant variations in the environment that are more significant than the tiny effect you are trying to measure.

There are many statical tests that can be used in these situations and have been honest over the years for use in scientific research.

For example, chi-squared and if you can quantify the error distribution and it's normal, student's t-test.

Well, more accurately you could chart a centile, but it would hide the very latency patterns you're looking for. The outliers in a given time bucket are the important data. If you're looking at one bucket, it could be useful to do a centile (though 99th is not really high enough to be meaningful), but if you're charting centile buckets, then you're missing the real latency spikes.

For all we know, KCM/KTLS generates the highest peak latency, but fewer times per bucket. That difference would completely change the interpretation of these data. If userspace data frame handling produces higher 99th percentile latency, that does not tell us anything about its maximum latency. Further, if we're looking at the 99th percentile, 1/100 data frames have latency higher than that percentile, this will happen hundreds, thousands, or millions of times per second depending on your interface bandwidth and typical frame size!

>> The paper reports a relatively small performance improvements. But I suppose if you're Facebook (they and RedHat implemented this) and performance improvements are measured in millions of dollars you care a lot about 7% less CPU utilization.

You think 7 percent is a small performance improvement? IMHO it's rather large. I take the time for much smaller than that.

To understand why, let me ask a question do you think doubling performance is a good thing? In other words a 100 percent increase in throughput or a 50 percent reduction in execution time. I hope your answer is yes because a lot of people think so. So what's 7 percent? Well imagine there are 7 different areas of your code that could be improved with each one contributing 7 percent. That's 49 percent or about what we'd call a significant improvement. Fixing up a number of single digit things adds up. But there is a more sinister side to it. Suppose you wanted to let those 7 little things go, and look for the big fish - the one that's going to get you 50 percent all in one shot. Guess what? It doesn't exist. Why? Because these little things add up to about half of your time, so to find the big one would imply taking the execution time for the rest of the code down to zero, because that's the only place there's 50 percent left. Unless you're doing something really poorly, there isn't a big win to be had. Performance comes from compounding the effects of many small optimizations.

If just moving TLS from the application to the kernel gives 7 percent - presumably by eliminating overhead - that's a good thing. Even if it's just 7 percent of time in communication which is only a portion of application performance, it's still a good thing.

--- note, this is a comment about performance and says nothing about tradeoffs in security by putting TLS in kernel ---

>> You think 7 percent is a small performance improvement?

I think a 7% reduction in CPU utilization for TLS is small given the vast bulk of systems that spend most of their time IO bound on network interfaces and/or storage and will see little to no improvement in throughput. I allowed that Facebook may see this as an important improvement given their scale; it could be worth millions of dollars in power and/or hardware.

>> that's a good thing ... it's still a good thing.

Sure, it's good. I didn't characterize anything as "bad," merely "small."

There's actually a notable security advantage to this. A userspace process can do key negotiation, hand the symmetric keys to the kernel, and then have an ordinary file descriptor, with the kernel handling encryption. Which means you could then pass that file descriptor to a separate process, let it read and write data, and not give it access to key material at all.

Can't you already do all of that with a userspace implementation?

That's pretty much what stunnel does.

Somewhat, if you pass a pipe around, but that also requires you to keep a separate process around, and shuttle data through two address spaces. With this, you can have a process do the negotiation, arrange a KTLS file descriptor, and then exec the new process.

Couldn't you open a TCP connection, do the handshake, then pass just the session key to the new process, along with the handler to the TCP connection?

You could, but you lose out on a nice interface. Kernel implementation allows you to not care that you're actually using SSL. It's a normal socket and polling / waiting / data transmission works exactly the same as without SSL. Handing over a session key means you still have to include an SSL lib.

This is likely to end up being supported in systemd soon (I'm guessing) to get an SSL socket activation.

Not really - if a control frame is received the KTLS file descriptor will report an error and disassociate itself from the TCP file descriptor, at which point you're supposed to transition back to using the userspace TLS library. You'll also need the userspace TLS library to do a clean shutdown.

In what scenario is it an advantage to be able to encrypt/decrypt data with a certain key, but not know the key directly?

Any scenario where the key is not ephemeral and you're handling client's input (can expect exploitation). If you have a good enough separation/sandboxing, any exploit wouldn't be able to steal the encryption key, or other private data, even if the exploit worked.

For handling encrypted secrets, this is popular as a HSM idea. You authenticate to a black box which does the crypto for you, but the key can't be extracted. Sometimes HSMs even have a physical tampering / self destruction protection.

So a scenario where the key is long-lived, but the process communicating is transient? I suppose that makes sense, although I struggle to imagine the use case.

SSL, certificate signing, configuration decryption, anything Amazon SSE is used for, ...

This only does the handling of TLS data frames in the kernel. All the handling of TLS control frames, including the handshake, is still done in userspace by an ordinary TLS library.

This means that kernel is only doing some symmetrical encryption and simple framing (which are both the same kinds of things it was already doing in other application domains).

This was my first thought as well. Looking through the repo [1] behind the paper the people behind it seem to have some crypto chops. A lot more can go wrong in kernel space, but the performance improvements might prompt the big players (Google, FB, etc.) to invest some time into locking it down.

Also, it seems that only particular parts of the stack are handled by the kernel. Per the README:

The socket does data transmission, the handshake, re-handshaking and other control messages have to be served by user space using appropriate libs such as OpenSSL or Gnu TLS.

[1]: https://github.com/ktls/af_ktls

Short answer, if all it does is symmetric cipher encryption/decryption it's probably not a big deal. The kernel already does crypto. Just keep the x509 parsing and handshake out of kernel.

> x509

Mmmm, I'm afraid I've got some unfortunate news for you :)

Too bad you got downvotes. It appears that kernel and/or kernel module signatures use x509 certs. https://access.redhat.com/documentation/en-US/Red_Hat_Enterp...

Thanks for the heads up.

Windows Server also moved HTTP to the kernel!

and since it was introduced (back in 2003), there were only 2 exploits discovered! (one RCE, one DoS)

So did Linux, once, with the Tux in kernel webserver (now removed).

Reply on news.ycombinator.com