GameNetworkingSockets – Reliable and unreliable messages over UDP
github.com
github.com
Unreliable messsages are for ones that are superseded by later messages. There's no point in retransmitting; you always want the latest object positions.
The big problem is message priority. Games need a low-latency, low-traffic channel for updates and a high-latency, high-traffic channel for assets. It's tough making this work on the open Internet. End to end QoS just isn't widely available. So using much less than the full available bandwidth is needed to keep intermediate FIFO buffers from filling up. This is related to the "bufferbloat" problem - network devices now all have lots of RAM, and if they have FIFO queuing and less output bandwidth than input bandwidth, they will build up huge queues that generate huge latency.
For QoS to work in the wild, there has to be some throttling or incentive to discourage too much high priority traffic. You'd like to have under 5% of your traffic at high priority. It's hard to make this work in the public Internet.
It's supposed to be a consistent-eventually system; the world doesn't stop just because of inter-region movement. So there's temporary inconsistency, resolved correctly most of the time. It's rather ad-hoc; this is 1990s technology. Also, the client is over-involved in region crossing. The client isn't trusted, but it's involved in the handoff of avatar control from one region to another. This was probably a design mistake - the communications rerouting and virtual object movement should have been separate.
Almost everybody else evaded that problem. Sansar has bigger regions (up to 4km on a side, but usually much smaller) and no region crossing at all. It's a simpler client/server architecture. None of the major successors to SL (High Fidelity, Sansar, SineSpace) offer a full connected virtual world. They're more about "experiences" - go in, visit pre-built world, leave. This simplifies their scaling problem.
Second Life is an alternate universe to the World Wide Web. There's much the same functionality - chat, voice, graphics, text - but the approach is completely different. The web is mostly stateless. SL is all about state. Everything is persistent, including the state of executing scripts. So most of the tricks used to scale web applications don't work.
Improbable supposedly offers large large multiserver regions but I don’t know if there is any user generated content platform built on top of it.
It’s totally legitimate to provide QoS based on open protocols, or even for particular classes of traffic. Net neutrality comes into play where providers start prioritising traffic based on the remote service provider.
As a separate issue, some providers are regulated (see Comcast) due to other laws. These Title II providers have additional rules that prevent them from making decisions on how. However, if they made such decisions on quality of service (rather than for profit gain, as they have in the past), it would be legal. The problem is that as a monopoly they have no reason to do that.
The server software is closed source, but the OpenSimulator project [2] exists as a third-party reimplementation.
[0] http://wiki.secondlife.com/wiki/Linden_Lab_Official:Second_L...
µTP works OK. The trick is to precisely measure latency (these UDP packets have timestamps in them with 1µs resolution), send enough data to achieve good bandwidth, but just enough to not fill these FIFO queues in both endpoints, and in various routers between them.
I understand that this is primarily derived from a gaming platform, but is there anything that makes this useful specifically "for game" and not networked applications in general?
I would argue that bandwidth-sharing fairness with stuff going over UDP is easy: Just start dropping packets when pipes get full.
Most updates are going to be pretty small. It's not like game developers want the user experience of their titles to be bad, after all.
You’re not consindering a server, the bandwidth is very high on the backend and does saturate links. You run many servers per physical or virtual machine due to cost, so you can have 1000s if players connected over a single network path.
At the datacenter level you're making sure that the bandwidth you bought from providers is sufficient (and ideally, redundant), and that you can shift load from one area to another if necessary. You can buy this capability from AWS or Azure, or build it yourself in many different ways.
In fairness that was probably a crucial component of the Internet's success, but you don't get the good w/o the bad.
But re: "this will result in bandwidth hogging", routers will just drop your packets if you oversaturate them, so that's not a worry in practice. Well, it is for developers but they use rate scaling algorithms in those cases.
I haven’t worked with either but did review the RTP spec, and many of these features appear similar.
RTP also has some multicasting features built in for broadcast delivery, which can be useful in certain contexts.
Edit: the reason I ask is that RTP is getting wide scale deployment and testing as it’s being used for WebRTC.
The gain here is that you typically get the latest information as fast as possible. The downsides, of course, are that there's a pretty steep decline on connections that are even a little high latency or lossy. UDP drops packets even on wired LAN, for example. Practically all games use this method, and they all have lots of smoothing and prediction tech to make things seem like you're getting constant position updates -- which you almost certainly aren't.
You can also use this for data that doesn't matter -- although if it doesn't matter you should question why you're sending it in the first place.
In terms of actual implementation, protocols that does this needs to keep state, regularly notify the other side what they've received, and retry if packets appear to have been lost. Doing it 'simple' is easy, maximising efficiency/performance is harder.
The other answer gives a good example of when you don't actually want to fix the issues you mention.
By its own admission, the project says: "The reliability layer is a pretty naive sliding window implementation". A bit scary if you really want to have guarantee message is delivered.
Also scary part: "Our use of OpenSSL is extremely limited; basically just AES encryption", "we do not support x509 certificates". OpenSSL is difficult, coming up with your own key exchange is likely problematic.
So I'm still searching for the answer as to why we need to do this? What's the market for this?
P.S. Yes, Valve being the author certainly brings some "reedeming value" to this!
Back-of-envelope calculation: You can get about 50,000 client connections at 50 updates/sec of 500 bytes each on a 10Gbit link. Four ports, double redundancy gets you 100K clients on a server. Yike -- that's wayyy more clients than you want on a single server (you almost certainly run out of server-side CPU for game simulation and so forth before you run into bandwidth issues). A dual 40Gbit networking card is pretty cheap, but you'll run into CPU load issues trying to feed that card enough traffic -- it's frankly plenty hard to do that even when you're not doing game computation.
You can probably run all of your servers on 1Gbit copper for under $100 / port. There are better ways to wire things up, but I've done this in the past and it's worked fine.
Capital outlay for sufficient server bandwidth just isn't a big deal.
[edit: back-of-envelope calculation low by a factor of 10 :-) ]
I also suppose not everyone knows how to terminate fiber and I've done it so much it's become second nature.
A "naive sliding window implementation" is the basis of reliable delivery in TCP. I think all they're saying here is that they don't have concept like selective ACKs. But SACKs are basically an optimization, not an issue of correctness.
Unless you’re sending executable and/or private content down the pipe, it’s rarely an issue for game networking stacks to eschew secure exchange entirely.
Although if a friend and I met up in a desolate place where he whispers me the secret, I'd argue the exchange was secure without encryption. So if you can exchange a message without anyone else knowing, encryption isn't necessary
As others have pointed out, things like Nagling and exponential backoff are not wanted or needed in gaming. Any design compromises made in the name of retrying delivery of old data are bound to be counterproductive.