Hacker News Reader

Comment by zinekeller | Hacker News Reader

As GP said:

> if you want reliable data transfer over UDP you have to basically implement almost everything TCP does on top of it.

Even QUIC has an ACK-like signal so you will never avoid it as otherwise you will deal with the two generals' problem (https://en.wikipedia.org/wiki/Two_Generals%27_Problem). The reason QUIC is more efficient than TCP is simply due to the fact that TCP can't evolve further due to middleboxes (called ossification, https://en.wikipedia.org/wiki/Protocol_ossification), not that TCP is inherently bad.

View on HN

Thanks for the explanation, I didn't know that about QUIC. But QUIC is trying to implement reliable delivery for applications, which makes sense as a general purpose protocol. but if I were doing a file transfer app/protocol these days for example I would send hash+Mac+metadata reliably and then check chunks of the file (unreliable+encrypted UDP)as they are decrypted (have to anyways for integrity checking as part if decryption), and peform reliable retransmission request when decryptiom fails. That's what I meant by application layer, application and use-case specific protocols. but perhaps general purpose protocols are still better (I am not sure, I was asking in my post, sorry for not being clear)

Then you would generally try to look at probabilistic reliability. One way systems like digital broadcasting requires probabilistic reliability so a lot of research was done here. Digital broadcasting has produced a lot of answers in this area, as you need to deal with both data corruption and missing data. On the data corruption side, you would want to implement a forward-error correction (FEC, https://en.wikipedia.org/wiki/Error_correction_code#Forward_...) to allow recovery of corrupted packets, while on missing data it can be mitigated by fountain codes (https://en.wikipedia.org/wiki/Fountain_code) that is basically FEC but deals with missing pieces instead of incorrect data. This of course does not guarantee 100% delivery but in practical terms this is humanly reliable.

Thank you, that certainly gives me a lot to think about in this area.

You could build this as an application mapping over QUIC. Within a single QUIC connection use reliable streams for the metadata delivery, and unreliable datagrams [1] for the payload. The benefit of a single connection is shared security context and shared fates of different datatype transfers.

[1] - https://www.rfc-editor.org/rfc/rfc9221.html

Reply on news.ycombinator.com