Hacker News Reader

Comment by nh2 | Hacker News Reader

What does "high performance" mean here?

I get 40 Gbit/s over a single localhost TCP stream on my 10 years old laptop with iperf3.

So the TCP does not seem to be a bottleneck if 40 Gbit/s is "high" enough, which it probably is currently for most people.

I have also seen plenty situations in which TCP is faster than UDP in datacenters.

For example, on Hetzner Cloud VMs, iperf3 gets me 7 Gbit/s over TCP but only 1.5 Gbit/s over UDP. On Hetzner dedicated servers with 10 Gbit links, I get 10 Gbit/s over TCP but only 4.5 Gbit/s over UDP. But this could also be due to my use of iperf3 or its implementation.

I also suspect that TCP being a protocol whose state is inspectable by the network equipment between endpoints allows implementing higher performance, but I have not validated if that is done.

View on HN

Aspera was/is designed for high latency links. Ie sending multi terabytes from london to new Zealand, or LA

For that use case, Aspera was the best tool for the job. It's designed to be fast over links that single TCP streams couldn't

You could, if you were so bold, stack up multiple TCP links and send data down those. You got the same speed, but possible not the same efficiency. It was a fucktonne cheaper to do though.

> I get 40 Gbit/s over a single localhost TCP stream on my 10 years old laptop with iperf3.

Do you mean literally just streaming data from one process to another on the same machine, without that data ever actually transiting a real network link? There's so many caveats to that test that it's basically worthless for evaluating what could happen on a real network.

Yes. Why?

To measure other overhead of what's claimed (TCP the protocol being slow), one should exclude other things that necessarily affect alternative protocols as well (e.g. latency) as much as possible, which is what this does.

It sounds like you're reasoning starting from an assumption that any claimed slowness of TCP would be something like a fixed per-packet overhead or delay that could be isolated and added back in to the result of your local testing to get a useful prediction. And it sounds like you think alternative protocols must be equally affected by latency.

But it's much more complicated than that; TCP interacts with latency and congestion and packet loss as both cause and effect. If you're testing TCP without sending traffic over real networks that have their own buffering and congestion control and packet reordering and loss, you're going to miss all of the most important dynamics affecting real-world performance. For example, you're not going to measure how multiplexing multiple data streams onto one TCP connection allows head of line blocking to drastically inflate the impact of a lost or reordered packet, because none of that happens when all you're testing is the speed at which your kernel can context-switch packets between local processes.

And all of that is without even beginning to touch on what happens to wireless networks.

Somebody made a claim that TCP isn't high performance without specifying what that means, I gave a counterexample of just how high performance TCP is picking some arbitrary notion of "high performance".

Almost like it makes the point that arguing about "high performance" is useless without saying what that means.

That said:

> you're not going to measure how multiplexing multiple data streams onto one TCP connection

Of course not: When I want to argue against "TCP is not a high performance protocol", why would I want to measure some other protocol that multiplexes connections over TCP? That is not measuring the performance of TCP.

I could conjure any protocol that requires acknowledgement from the other side for each emitted packet before sending the next, and then claim "UDP is not high performance" when running that over UDP - that doesn't make sense.

> I gave a counterexample of just how high performance TCP is picking some arbitrary notion of "high performance".

You gave a vacuous example of a measurement of packets going nowhere. When zero work gets done, you might as well call it infinitely fast.

No, the example exercises the full TCP protocol including all its logic.

There have been plenty of examples of protocols where that is far from infinitely fast and does not scale even on localhost, e.g. with OpenVPN or any protocol that requires full acknowledgements from the other side before sending more.

High performance means transferring files from NZ to a director's yacht in the Mediterranean with a 40Mbps satellite link and getting 40Mbps, to the point that the link is unusable for anyone else.

UDP by itself cannot be used to transfer files or any other kind of data with a size bigger than an IP packet.

So it is impossible to compare the performance of TCP and UDP.

UDP is used to implement various other protocols, whose performance can be compared with TCP. Any protocol implemented over UDP must have a performance better than TCP, at least in some specific scenarios, otherwise there would be no reason for its existence.

I do not know how UDP is used by iperf3, but perhaps it uses some protocol akin to TFTP, i.e. it sends a new UDP packet when the other side acknowledges the previous UDP packet. In that case the speed of iperf3 over UDP will always be inferior to that of TCP.

Sending UDP packets without acknowledgment will always be faster than for any usable transfer protocol, but the speed for this case does not provide any information about the network, but only about the speed of executing a loop in the sending computer and network-interface card.

You can transfer data without using any transfer protocol, by just sending UDP packets at maximum rate, if you accept that a fraction of the data will be lost. The fraction that is lost can be minimized, but not eliminated, by using an error-correcting code.

> perhaps it [..] sends a new UDP packet when the other side acknowledges the previous UDP packet. In that case the speed of iperf3 over UDP will always be inferior to that of TCP

It does not, otherwise it would be impossible by a factor ~100x to measure 4.5 Gbit/s as per the bandwidth-delay calculation (the ping is around the usual 0.2 ms).

With iperf3, as with many other UDP measurement tools, you set a sending rate and the other side reports how many bytes arrived.

You are right.

It is a long time since I have last used iperf3, but now that you have mentioned it I have also remembered this.

So the previous poster has misinterpreted the iperf3 results, by believing that UDP was slower, as iperf3 cannot demonstrate a speed difference between TCP and UDP, since for the former the speed is determined by the network, while for the latter the speed is determined by the "--bandwidth" iperf3 command-line option, so the poster has probably just seen some default UDP speed.

But I am "the previous poster".

And I am quite sure that UDP is slower, becuase I increase `--bandwitdh` until throughput stops increasing, which is at 20% of TCP's speed.

Reply on news.ycombinator.com