Hacker News Reader

SCTP [1] is there to provide a reliable message based protocol. And it does work inside a datacenter. The issue is outside the datacenter: it doesn't work reliably across the Internet due to middle boxes dumping anything not TCP or UDP...

But inside a controlled environment like a datacenter, it works. It's been used in the telecommunication world to carry control messages in the radio access and core networks for example. So it's been tested at scale for critical applications.

[1] https://en.wikipedia.org/wiki/Stream_Control_Transmission_Pr...

> The only kinda-sorta exceptions I know about are HTTP/1.1 and telnet. But HTTP/1.1 is still a message oriented protocol; just with file-sized messages. (And even this stops being true with http2 anyway).

No, HTTP/2 and QUIC do not change the semantics of HTTP.

Also, you can have endless streams with HTTP/1.1: just use chunked encoding to POST/PUT and use Range: bytes=0- and chunked encoding for GET and chunked encoding for POST response bodies. In HTTP/2 there's only the equivalent of chunked encoding -- there's no definite content length in HTTP/2.

Correct me if I’m wrong, but doesn’t h2 still break up requests and responses into smaller message frames in order to do multiplexing?

Those message frames are what I’m talking about - as I understand it, they are, yet again, a message oriented protocol layered on top of tcp.

You will always need _some_ framing if you're dealing with bulk data.

If it's not bulk, then you don't need framing if everything can fit into one datagram / whatever transfer unit provided by the transport, but the transport itself will need some framing, especially if it will need to support any kind of fragmentation.

If it's not bulk and it doesn't fit in a datagram / whatever transfer unit provided by the transport and the transport doesn't do fragmentation, then you have to do framing yourself, and then you have a sequencing problem, and so on, and you quickly re-invent parts of TCP but at the application layer.

Basically, it seems inescapable that the Internet is based on packets, and that packets are limited in size, and so application protocols have to be smeared onto packets.

Things are only ever trivial when you're doing request/response protocols with always- or mostly-small requests and responses. The moment you need anything that doesn't fit in the path MTU minus overhead, you need framing.

So I don't think that an octet stream abstraction is quaint and obsolete.

Yes, and even HTTP/1.1 does that. I forget if telnet does something similar, but I suspect it must because it can send control data. The FTP protocol, and the BSD r-command might be the only ones that truly do no additional framing (FTP for data connections, r-commands post-login).

Once chunked encoding is in the picture, even HTTP/1.1 sends messages, not streams, under the hood.

Correct.

I don't think so, or we disagree on the meaning of words. Chunking is not message-oriented in the same sense that UDP is.

With chunking, you basically just insert markers into the stream; this does not imply by any mean that the stream has been split in multiple messages - as a matter of this is taken care of by the lower levels of the client/server, the middle/higher levels certainly don't want to deal with it. This only is a perverted solution to the problem of dynamically generated "messages" (mainly HTML pages), that has been further perverted to implement gruesomely message-oriented "protocols" (Comet and others, IIRC).

UDP, on the other hand is based on datagrams. They can be split into smaller packets on the wire but they are reassembled at the network stack level so no program can even see it happened unless they insist on it.

Websocket is much closer to a message oriented protocol over a streaming pipe than HTTP chunking is.

> I don't think so, or we disagree on the meaning of words.

Well, when there's four people in the conversation, that happens.

u/josephg's complaint at the top of this thread is that people use TCP but still have to add framing in their application protocols. u/josephg said something to the effect of how few protocols do no framing and mentioned HTTP/1, but even HTTP/1.1 w/ chunked transfer encoding adds framing, and even HTTP/1.0 w/ definite content-length also has framing (CRLFs) for the request headers themselves, and effectively frames bodies with CRLF at the start and EOF at the end.

Chunks are definitely not datagrams, just as TLS records aren't either, and just as TCP segments aren't either. But they have framing, which u/josephg complained about.

Framing of some sort is unavoidable. My point, besides the inevitability of application-layer framing, is that a datagram- or message-oriented transport won't make things trivial for apps anymore than TCP did.

> In my opinion, the real problem is the idea that "everything is a file".

Files are just an indexed list of bytes that can represent anything. Think of them as objects.

> Byte streams aren't a very useful abstraction. "Everything is a stream of messages" would be a far better base metaphor for computing.

I don't understand this. A stream of messages is a stream of bytes. Its bytes all the way down.

Because most protocols can handle message loss, with retransmit and proper ordering ? And we haven't started to talk about congestion yet… TCP is useful, and while I'd like to see a message get rid of one (or more) of those constraints to go with a a custom protocol, I feel like they'd be re-implementing the features in the end because these are very useful properties to have…

Edit: the proposal in the article is actually quite, sensible, but requires redesigning your apps… And I'd like to see how it performs: TCP is a hugely optimized beast (when it works well), with hardware offloads, kernel optimizations, etc.

Even TCP itself uses discrete messages under the hood :-)

Hah yes; although TCP frames can be arbitrarily refragmented and rejoined as they travel through the network before reaching your destination.

If you ever play an indie game which seems unusually janky over wifi, its probably because the code isn't correctly rejoining fragmented network packets at the application level. Ethernet is remarkably well behaved in this regard. Wifi is much better at shaking out buggy code.

Finding the right abstraction isn’t easy in a network stack. TCP’s is less useful for application logic but it reflects the way the protocol works internally. Bunch of Bytes goes in on one side, Bytes stream out on the other side in chunks whose size depends on congestion, physical layer and other facts. A message based API hides these facts or leaks them depending on how you look at it.

What’s the issue with fitting messages in streams?

One of the issues I can think of is head-of-line blocking [0].

If you're sending messages of different priorities over the same channel, an error in sending a low-priority message, high-priority messages will have to wait until the low-priority message is properly re-transmitted.

[0] https://en.wikipedia.org/wiki/Head-of-line_blocking

https://en.wikipedia.org/wiki/Head-of-line_blocking

Yes, but this need happen only when your data center network is congested, which is hopefully rare and is relatively cheap to fix by adding capacity. And in congestion cases you need TCP's back off ability.

Getting rid of head-of-line blocking also makes messages happen in order. Making messages (sometimes) happen out of order would drastically increase implementation complexity for a lot of apps.

I don't bother implementing that crap anymore. I just websockets which are message oriented out of the box.

What do you make to require implementing or making network protocols?

Multiplayer video games, realtime updating webpages, database bindings, sync protocols (I play around around with CRDTs a lot), p2p distributed systems, whatever really!

Implementing a wire protocol seems to come up about once every couple of years. And for context, I've been programming now for about 30 years.