I wonder why SCTP isn't more widely-supported. It seems like a lot of Internet traffic is message-based rather than stream-based and would benefit from a more appropriate protocol.
It really is unfortunate that we ended up with UDP and TCP as the only choices. There are so many more things you might want outside of "unreliable packets" and "automatically recovered streams."
* Most ISPs don't mangle UDP, and if you care enough about perf to be using UDP in the first place, you can afford to figure out who mangles UDP and fall back to TCP for those networks. It's a very small number of networks.
* Any TCP/SCTP/etc inspired protocol you can think of can be implemented on top of UDP. Its header is only eight bytes long and it adds minimal overhead. UDP socket semantics are simple enough that you can treat them as a slightly smarter IP socket and build your protocol as if you were running directly on top of IPv6.
* There's a huge advantage to being able to implement your protocol in userland since you get fine grained control over congestion and connection logic. There's a huge disadvantage too, since it's actually hard to implement that stuff yourself. But for the people who are large enough to build their own protocols (FB, Google, etc), it's worth it.
* You can even encapsulate SCTP inside UDP to punch through middleware firewalls if you want to. I don't know of anyone doing that for real in the wild, but there's a RFC for it (RFC 6951).
The fine grained control over congestion and congestion logic is cool, sure. But more important in my mind is the fact that there's a cross-platform API available and you can rapidly make changes to how the protocol works without needing to require a specific kernel version or a specific operating system.
There's a whole lot of really cool frameworks for SDN to allow people to work at lower levels, but that's the problem - a standard API that everyone can expect to be widely available hasn't really emerged yet.
Aren't WebRTC data streams implemented with SCTP over UDP?
E.g. "ICE" provides a negotiation protocol to make a couple arbitrary machines able to talk to each other directly, including the NAT hole punching part of things. In the limited % of cases where there's no known way to punch a hole through NATs, an automatic fallback (using a relay server) is provided as a part of the protocol.
It's not elegant, at least we don't have everybody re-inventing the wheel. And, good results when ICE is used can become part of the criteria now used to judge the quality of NATs and other evil middlebox devices.
Anyway, I'd love to be able to pick and chose among:
1. Packets versus streams.
2. Reliable versus unreliable.
3. Immediate delivery, or delivery in the same order as data was sent.
Right now, we have UDP, which is packets, unreliable, immediate, and TCP, which is streams, reliable, delivery in order of transmission. But all eight possible combinations make sense (albeit some more than others), and it's unfortunate that we have to reimplement stuff that already exists if we don't fit within the two choices we have.
And if we're going beyond what TCP provides, I'd really like to see:
4. Choice of ACK versus NACK based loss notification.
5. Forward error correction.
6. Multihoming.
#5 should really be done at the framing layer, otherwise your frame delimiters can also be corrupted. It's also very tied to physical layer and probably better off not being configured in platform agnostic software.
No opinion on #6, I've not deal with it much AFAIK.
I also missed one, which is hard to get right: rate limiting. Especially if you want to coexist with TCP rather than drown it out (or be drowned out by it).