Hacker News Reader

Container formats increase size. Now for video that doesn't matter much because it doesn't move the needle. For images a container format could be a significant percentage of the total image size.

Yes, I focused mostly on the lack of benefit, but even for a single stream, size is another important cost.

> The alternative would be unnecessary demux on the server side, or the client unnecessarily downloads irrelevant streams.

HTTP file transfer protocols support partial downloads. A client can choose just to not to download irrelevant audio. I think most common web platforms already work this way, when you open a video it is likely to be in .mp4 format, and you need to get the end of it to play it, so your browser gets that part first. I am not entirely sure.

I believe mp4 files can be repackaged to put the bookkeeping data at the front of the file, which makes them playable while doing a sequential download.

That metadata is usually put around the end of the file for compatibility reasons, but one can use ffmpeg's `-movflags faststart` option to move it to the beginning (very common in files that are meant to be served on the web).

> You typically serve individual and separate streams of video, audio, and text, sourced from separate files, and your player/browser syncs them.

That's one school of thought. Some of the biggest streaming providers simply serve a single muxed video+audio HLS stream based on bandwidth detection. Doesn't work very well for multi-language prerecorded content of course, but that's just one use case.

That's true, but my understanding is they serve a specific mux for a specific bandwidth profile, and serve it by just transmitting bytes, no demux required. I didn't mean to imply that wasn't a common option. I only meant to say I don't think a common option is to have a giant mux of all possible bandwidth profiles into one container file, that has to be demuxed at serve time.

My understanding is that YouTube supports both the "separate streams" and "specific mux per-bandwidth profile" methods, and picks one based on the codec support/preferences of the client.