Excuse my ignorance, and I'm sure 2 seconds is probably an engineering feat, but I'm genuinely curious. What is it that prevents latency to go as down as a few hundred ms (pretty much close to and IP round trip) ?
1) If you want very low latency, any network jitter or delays will cause pauses on the viewer side and "skips" when the feed catches up after a brief dropout. This is fine for video chat, where a little blip doesn't interrupt the experience. For live streams with 30k+ viewers, it's pretty annoying and very noticeable if the audio cuts out or skips. A 2-3 second window is typically large enough to paper over any jitter or retransmits due to packet loss between the broadcaster and Twitch servers.
2) Transcoding can be done with very low latency, but it's harder to scale horizontally and uses more bandwidth than if you give yourself a few hundred ms of buffer. Larger buffers enable better compression. Transcoding is needed if you want to stream to mobile, web, etc. in multiple formats, bitrates, or resolutions.
3) Chunked HTTP content is much easier to serve than RTMP or WebRTC-style content. You can use nginx or drop your content on a low-cost CDN. The caveat is that chunking generally introduces latency unless you do something fancy such as streaming chunks as they're being written to disk.
Source: I designed the video streaming network that Twitch was running when Amazon acquired it. More info here http://highscalability.com/blog/2010/3/16/justintvs-live-vid...
The most cost-effective way of delivering video is using some form of HTTP streaming (like HLS or DASH). In a nutshell, the player downloads a manifest that tells it where to find chunks of video, which are downloaded and cached in normal, commodity CDNs. Everything is stateless and is scaled like any other form of HTTP download. To do all of this you end up needing to transcode the incoming stream. All along the way through this process introduces latency, and for reference 20 seconds is perfectly normal HLS latency, so credit where credit's due, this is really impressive. One of my colleagues wrote about the state of low latency last year[1], and considered < 4 seconds to be "ultra low latency"...that's really rare, particularly among platforms.
You can get lower latency, of course, but typically that involves stateful connections. All of that cheap commodity stuff that comes with HTTP streaming above goes away, and scaling to a large number of viewers can get extremely costly (and operationally difficult).
[1] https://mux.com/blog/the-low-latency-live-streaming-landscap...
Yep! I think Twitch can sometimes be at 4 seconds or less for what it's worth, but yes, that delay is real. It's generally not really noticeable because the communication is totally async; the streamer is doing other stuff, finishing other thoughts, etc, then can get to messages as they see them.
> can you explain what you mean by “stateful”?
Sure! I was talking about what kind of connection is necessary to deliver the video. A lot of real-time video solutions require stateful connections to the client, which means that once the client connects, the connection is kept open, and video data is streamed via that connection. Common examples on the web are things like WebSockets and WebRTC, but it gets really expensive because you basically have to maintain a persistent connection with every single viewer and it makes it impossible (or extremely difficult) to do any kind of meaningful caching.
Stateless connections on the other hand are most of your common HTTP requests. The client asks for a resource, and gets it back. No prior connection setup is required, servers, networks, whatever can change between each request and everything will merrily chug along, which makes it much easier to scale.
This is why we built Pushpin, to make it easier to handle stateful connections at scale. The project is mostly intended for moving application data, but it does work for media streaming too. See our live MP3 demo [1]. The backend runs GStreamer in a loop to produce the audio, and has no awareness of client connections. Pushpin moves the bytes and knows nothing about audio or codecs.
Currently, IVS configured for "ultra low latency" is using HLS segments that are three seconds long. The client tries not to buffer more than one segment, so on a good network connection you'll see ~4 seconds of latency.
In theory, you could start playing the video while you're still downloading the first segment. That's how you'd get ~2s of latency. But the AWS player doesn't actually do that. And for good reason. These are TCP connections, so if there's any packet loss at all, you'll have to either buffer or skip the segment and change bitrates. Starting the video and then immediately buffering is a pretty poor user experience.
This is pretty easy to test. I just did, twice: streaming from OBS on my desktop and then directly from our compositing servers in the cloud. In both cases the latency was ~4 seconds.