Comment by ANTSANTS | Hacker News Reader

Comment by ANTSANTS | Hacker News Reader

Parent Full thread

Games and streaming audio/video aren't common scenarios on the internet?

Don't be distracted by latency/packet loss/bandwidth configurations, the main problem here is that TCP was never intended to serve the needs of soft-realtime communication. It's not relevant to deterministic lockstep, but for most realtime problems (I'm thinking of multiplayer FPS, but you can think of Skype if that's not "serious" enough for you), if you miss a packet of data, it might as well have never existed. Better to hear blips and glitches on a phone call over a shitty connection than to start adding seconds of latency.

You missed the point about deterministic lockstep as well, which is intended for scenarios where trying to transmit i.e. the entire state of a virtual army per tick is prohibitive. Designing the program to behave deterministically so that you can send only the player inputs per tick is the exact opposite of an "extremely high available bandwidth" solution.

Audio/video streams are a completely different problem - lost data is just lost. This article is discussing a very specific problem where no data loss can be tolerated.

The solution described only works when very high bandwidth is available, because it aggressively sends duplicate copies of data until acked, using on average 2-15x more bandwidth and 120x more bandwidth in the worst case (numbers taken directly from the article).

If you happened to have that kind of spare bandwidth available then yes, this is an excellent solution. When do you have high latency, packet loss, and bandwidth overprovisioned by a factor of 15?

Deterministic lockstep is primarily used in RTS games, where the only data that is transmitted per tick are tiny commands like "player 1 is constructing a war factory at this location" or "player 1 told unit 6 to move to this tile." If we're ridiculously generous and say that a kilobyte of commands are sent per tick, then 120x bandwidth would be 120 kb/tick. 10 ticks/s is a common RTS tickrate, so that's about a megabyte per second. In practice, far, far less bandwidth would be used than this, given that this architecture scales down to 90's dialup connections (https://www.codeofhonor.com/blog/the-making-of-warcraft-part...), but even a megabyte/s is not a completely ridiculous amount of upload bandwidth for much of the first world (but sadly not the USA).

> Audio/video streams are a completely different problem - lost data is just lost. This article is discussing a very specific problem where no data loss can be tolerated.

I know that, which is why I separated deterministic lockstep from other solutions in my comment. You're missing my point, which is that soft-realtime problems aren't suited for TCP, and cannot be addressed by any single protocol because constraints differ.

You appear to be missing my point, which is that this proposed protocol responds to network congestion by increasing the amount of bandwidth that it uses by a factor of more than 10.

Consider what happens when many clients are using this protocol over a shared transit link and it approaches 80% utilisation, so starts to drop packets at a low rate: all the clients will start batching up larger amounts of history and sending them, which increases the utilisation, which increases the packet loss, which increases the amount of history each client sends... it's a destructive feedback loop that launches a DoS attack against the congested link. The absolute size of each individual client stream is unimportant, as we are merely assuming that the number of clients sharing a link is high enough to congest it. The last-mile user connection is not the worst problem here.

Network congestion is not the only reason for packets to be dropped. Wifi and mobile connections can be hampered by radio interference or have trouble transmitting through thick walls, hardware can be faulty, wires and fiber can be damaged, routers (especially home routers) can be poorly configured, cosmic rays can flip bits, etc. The internet is not perfect, and network programmers in a certain soft-realtime, low-bandwidth domain have noted that sending duplicates of their tiny packets instead of waiting around for ACKs results in an improvement in reliability and average latency at negligible cost. You're trying to extrapolate this idea way too far. No one is saying we should swap TCP out tomorrow with an alternative that works that way.

This isn't a theoretical, unproven suggestion he's recklessly advocating -- this technique is implemented in several games played by millions without your DoS scenario playing out. Games are typically low bandwidth applications -- they use that extra wiggle room to increase reliability and responsiveness with reliable real world success.

He's also written about flow control to properly handle congestion when it does happen: http://gafferongames.com/networking-for-game-programmers/rel...

I wonder if something like Aeron (https://github.com/real-logic/Aeron) would work.

To be clear, not all games are networked using the deterministic lockstep technique but most realtime games (FPS and so on) do use UDP to send time critical data, and implement some aspects of reliability, ordering and congestion avoidance within their own custom protocol.

Yeah sorry -- I should have been more clear. I didn't mean to imply most games use deterministic systems.

you're not correct. The amount of bandwidth is increased by a small constant factor, but still likely far less than doubling the total packet size.

The point is though that sending 15x more data is not an issue when you're only sending a few bytes per tick.

When optimizing, you always profile and optimize the bottleneck first. For a lot of people in a lot of situations, latency, not bandwidth, is the bottleneck, and that's exactly what this solution solves. Compared to e.g. Standard web usage, most games are actually very small amounts of data sent, but very high time criticality, and hence very sensitive to latency.

In the case of wifi and such this seems like a sensible approach, too bad its being applied end to end.

All that would be required IMO for this to be a perfect solution for time critical data would be if we had an IP packet header flag to indicate to routers that our packet is time critical, so if it can't be passed on immediately, don't buffer it like a TCP packet, just drop it. -- cheers

Reply on news.ycombinator.com