johnousterhout

I think you may misunderstand Homa's behavior. In general there's no reliable way to predict how long it will take to service an RPC, so Homa doesn't try. Homa does quick resends using the same timer that's used to detect packet losses (i.e. a few ms), so the results will be pretty much the same as what you described for an ack-based scheme. Of course, if an RPC takes more than a few ms to complete then there will be an unnecessary resend request, but I don't expect this to happen very often. And, it won't have a significant performance impact because the RPC is already consuming many ms of CPU time for its execution; a few microseconds to handle a resend is insignificant.

I agree that a transport cannot hide all possible failures and that at some point things have to get kicked back to the application. But if the only guarantee provided by the transport is byte delivery in one direction, that makes things significantly harder for apps: every app now has to implement its own timers and pings to detect server failures. This basically duplicates timers and retries already present in the kernel. I think Homa's approach makes life easier for apps. There's a clear and simple division of responsibility. Homa detects all failures, including both lost packets and server failures. It recovers from packet loss but not server failures. Server failures are reported to the app, and the app decides how to handle them.

And, Homa's approach cuts the number of packets in half in the common case of short RPCs with fast service times. With acks it takes 4 packets for each RPC: one for the request packet, one for its ACK, one for the response packet, and one for its ACK. With Homa there are only 2 packets in the common case: one for the request and one for the response. The response implicitly acks the request, and the next request acks the previous response.

Hmmm, I see now that QUIC can delay ACKs and piggyback them on subsequent data packets, so it looks like it can get similar efficiencies to Homa.

I agree that Homa holds Tx resources for longer than would be needed in an ack-based scheme (I don't see that as particularly problematic).

In your item 2, it's important for the receiver to know the message size not just for allocating Rx resources, but also for prioritizing its incoming messages.

The problem you discuss in item 3 (an intermediate link much slower than the host uplinks) seems like something that will only occur in a WAN. In datacenters the host uplinks are the slowest links, so I don't think this is an issue for Homa.

I'm not sure what you're getting at with your point 1. Any implementation of RPCs, over any protocol, faces the same timeout issues. If it takes a long time to service an RPC, the client will naturally wonder if something broke. Either the client has to send some sort of "ping" to make sure the server got the request and is still alive, or the server has to send "keep alive" packets to let the client know it's still alive. If most RPCs finish quickly then the ping interval should be short in order to detect packet drops quickly; but then a large number of pings will be required for RPCs that take a long time to finish.

I don't see anything particularly Homa-specific in these issues. A protocol like TCP that doesn't implement RPCs doesn't have to face these issues; it punts them up to the application, so the application has to deal with them. Homa implements all of this in the transport, so applications don't have to worry about it.

My knowledge of QUIC is pretty superficial, but it sounds like you are saying that congestion control is not a fundamental property of QUIC, so someone could potentially implement Homa's congestion control mechanism in QUIC. If someone were to do that, then of course I'd expect that to perform similarly to Homa.

However, this would create a conflict within QUIC. My understanding is that one of QUIC's primary use cases is WAN connections between browsers and Web servers. In this world Homa's congestion control mechanism does not work; it only works in datacenter environments with speed-of-light delays at most a few 10's of microseconds. Thus, QUIC would need to support multiple congestion control mechanisms, one for WAN and one for datacenter and choose between them as appropriate.

In addition, my understanding is that QUIC is stream-based, which would mean, as with TCP, the protocol doesn't know about message lengths. This makes it hard to prioritize short messages.

And, by the way, as much as you may prefer acks to grants, the grant approach is pretty fundamental to Homa's congestion control.

To me, QUIC and Homa were designed for very different environments. Homa will never compete with QUIC in the WAN, and I think it will be hard for QUIC to compete with Homa in the datacenter (Homa can specialize itself entirely for datacenter apps, whereas QUIC can't).

Just to set the record straight:

1. Homa detects and recovers from whole-RPC loss just fine; always has.

2. Homa does not have built-in encryption support (any more than TCP does), but encryption can be (and has been) layered on top pretty easily with DTLS. Isn't that what QUIC does also (layer encryption on top of UDP)?

3. Performance: I think you may be misreading the figure. For W4 or W5, Homa uses about 5 cores to get a total of 40 Gbps (20 Gbps in each direction), which is about 8 Gbps/core. I think you'll be hard-pressed to find a software transport that does much better than this (make sure you are measuring mixed workloads with a variety of message sizes; if you only send large messages then you can do much better (and so does Homa)). Unfortunately, all software transport protocols are resource hogs; in my opinion, transport implementation really should move to the NIC.

4. I would be interested to see measurements of QUIC performance running the same workloads as Homa. My understanding is that QUIC has all of the congestion control problems of TCP, so I'd expect it to perform like TCP under mixed workloads (i.e. 10-100x longer tail latency than Homa for short messages). Among other things, I don't believe QUIC uses switch priority queues, so it does not "solve basically every problem Homa is trying to solve".

Good point about using types. I should have mentioned that as a way to eliminate the block number problem in Section 14.1; I'll fix that in a future revision of the book.