Hacker News Reader

Comment by mmastrac | Hacker News Reader

Very cool. I hope we can start seeing WebAssembly-pluggable codecs in all of the video/audio paths in the browser. Imagine being able to turn on a brand-new codec (or select a customized codec for your particular use) in any browser that supports WebAssembly.

Two ends of a WebRTC connection could even negotiate codecs based on power/processing needs, with one sending a WebAssembly codec to the other as a fallback if needed ("hey you don't support this thing that I support in hardware, but you are plugged in, so burn some more electrons on your end").

WebAssembly SIMD [1] will definitely help with performance here.

[1] https://www.chromestatus.com/feature/6533147810332672

View on HN

Although there is much to be desired still on the software end, the molasses of the situation is hardware support for the new formats. You mentioned it briefly, but there's only so much you can do with a software decoders. H.264 may not be ideal or technically have the best performance, but most video cards and cpu's have dedicated hardware decoders.

Completely unrealistic, but it would be interesting to have on the fly reprogramable FPGAs on die that can be assigned specific tasks on demand. This of course has tons of risks, but the pace that hardware moves is painful.

H.265 support is a patchwork in hardware though, look at all the recent Allwinner chips, only 8 bit H.265 playback is supported, despite quite a few H.265 files using 10 bit colors.

Worse yet, most Chromecasts and TVs don't support anything newer than H.264. It will be years before H.265 or any new competitor sees hardware support in most homes.

H264 was the same way early on, with several chips doing baseline or main profile, but not high. It's why I said (at the time) that H264 HW support wasn't important... being able to do Bluray profile (a specific variant of high profile) was what was actually important.

Now with H265, Bluray 4k profile is important, ergo 10 bit support is mandatory, as well as display output itself being able to handle signalling for Rec2020 HDR10 over HDMI 2.x (or equivalent DP).

Probably due to H.265 having much more expensive licensing costs than H.264, given enough volume. That alone might make lots of players in the industry to just prefer ignoring the new codec (and joining a joint effort for developing a new codec with more favorable licensing terms...)

Yeah, VP9 will probably monopolize the market as the libre, royalty free codec. With Google and others boosting it, I'd be surprised if it doesn't win like Opus has in the voice codec arena for any new application.

> Completely unrealistic, but it would be interesting to have on the fly reprogramable FPGAs on die that can be assigned specific tasks on demand.

I've always wanted this! "Hardware accelerated" would become a thing of the past, because everything could hardware accelerate itself. It would probably be more like ram than a graphics card, though, and could probably be designed so each application could specify how many gates it wants.

> This of course has tons of risks

Like what?

> I hope we can start seeing WebAssembly-pluggable codecs in all of the video/audio paths in the browser.

Oh great, one more thing to help ad networks to crank CPU usage of my browser to the extremes. Hope I'll be able to also disable that through about:config. :/

> I hope we can start seeing WebAssembly-pluggable codecs in all of the video/audio paths in the browser.

I have a n00b question though. How would I display the video in the browser using webassembly ? I mean using webassembly, I can do the decoding of the bits in the stream. But how to display those bits in the browser ? Do I use a canvas and directly dump the pixel buffer there ?

Surprised to see the decoder was not a WebAssembly module.

I would assume (completely uninformed assumption, however) that WebAssembly without SIMD won't be performant enough to decode this in realtime.

We already have an enscripten'd decoder, built into a nice analyzer tool that lets you inspect the AV1 bitstream! But you're right, it'll need SIMD to decode at high resolutions in realtime.

[1] https://0x0.st/tQ

JavaScript doesn't have SIMD

JavaScript was once to have SIMD. ;- )

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...

Yep, but now just WebAssembly

They're kinda unrelated. I don't think WebAssembly has explicit SIMD operations yet? Then again, maybe WebAssembly could be delivered in a form which makes autovectorization straightforward for some subset of use cases.

There's a proposal for SIMD but it's not in any shipping browsers AFAIK.

https://github.com/WebAssembly/simd/blob/master/proposals/si...

Reply on news.ycombinator.com