How LCEVC works – A new approach to video compression | Hacker News Reader

How LCEVC works – A new approach to video compression

lcevc.org

88 pointsclouddrover34 comments

The marketing page here looks pretty great, and looks like an inventive approach. And yet, because it has MPEG's stamp on it, I'm terrified of the numerous licenses of which patent holders I would be beholden to in order to even use the darn thing at scale. My primary focus has been live independent creator sourced video for a few years now. You can certainly make up the licensing in volume for VOD content in the big player spaces, since you have a much better ratio of media to consumers, but in the market I'm the most interested in, this is practically a non-starter.

It is a technology mainly ( if not solely ) created by V-Nova, Standardise by the MPEG Working Group and licensed by V-Nova themselves [2] ( Not MPEG LA ). Basically free software and hardware implementation. But they charge content provider at a small per user / view fees and capped $3.7mn per year. Great for TV Broadcast or even Internet download ( as in torrent ). Not so great for free online streaming unless you are paid streaming services like Amazon or Netflix.

The new Brazilian SBTVD Forum [3] has chosen LCEVC along with VVC as baseline for their TV 3.0 codec.

The great thing about LCEVC ( on paper as I haven't tested it myself ) is how it reduce encoding and decoding complexity on baseline codec while giving you better quality. You are encoding a 1080P stream and upscaling it to 4K ( which your 8K TV will also upscale it to 8K LOL ). It helps with a lot of encoding or decoding complexity with latest gen Video Codec such as AV1 and VVC. I haven't checked if there are any latency issues though.

I only wish they provide a freemium mode for non-revenue generation ( no -ads ) usage on the web.

[1] https://www.v-nova.com

[2] https://www.v-nova.com/press-releases/v-nova-lcevc-licensing...

[3] https://www.digitalmediaworld.tv/broadcast/4118-brazilian-sb...

> You are encoding a 1080P stream and upscaling it to 4K

I mean, yeah, one way to reduce encoding/decoding complexity is to throw out 75% of the pixels. Personally, I would avoid this like a music service that decided 11 kHz was good enough for encoding and added pretend high detail on playback.

The article describes how the layered approach works, including that it can be used as a lossless codec if desired. The core idea is that the residual layer only needs to encode the errors after upscaling, which even in the lossless case can be more compact.

Thanks! That makes more sense now.

The article is not very clear. See one of the references for a better discussion.[1]

The key idea here seems to be to have a format which will play usefully on both mobile and 4K TVs. All players take the low-rez stream, and higher resolution devices also take the "enhancement" stream to fill in fine details.

[1] https://www.lcevc.org/wp-content/uploads/A-Review-of-Emergin...

I think the key use case is installed hardware decoders. So rather than replace a set top box to get it to decode a next generation codec, you can send it the previous generation codec, which the hardware handles, and then use limited CPU you have to 'upscale' that.

I don't understand why you would use this in a greenfield situation or in any place that is purely software. Since surely you could just build this into the new codec and get all the benefits?

Is this really that new? Its basically just using 2 codecs as a hack to get around the fact that hardware is hard to change and thus hard to make improvements to. So instead use one hardware implemented codec that is fixed, and a software one that can be more easily improved.

Surely someone has done that before?

I am doing something similar, just for the sake of it, on the PlayStation 1 of all things: a custom video codec that takes advantage of the console's hardware MJPEG decoder, but replaces the software side of the stock codec (normally in charge of decompressing the Huffman encoded bitstream into run-length tuples, which are then fed into the decoder) with a slightly more efficient compression scheme.

I'm also trying to leverage the GPU to post-process decoded blocks by e.g. blending them with the previous frame, on hardware that was never meant to support p-frames in the first place. The whole project is still in its early stages though, so there is nothing I can show off (yet).

There are several digital encoding schemes that add enhancement layers on top of an older, established base layer. It's a key concept of backwards compatibility. Not including cases where additional channels or domains of information are added (e.g. surround sound, 3D video), and only looking at cases where the perceptual quality of the existing channels or domains is improved (e.g. HDR, resolution), here are some examples:

* Progressive JPEG (1992)

* Spectral Band Replication (2001) in aacPlus -> HE-AACv1 (2003), mp3PRO (2001), WMA 10 Pro (2006)

* JPEG XT (2015) - enhancements on top of 1992 JPEG.

* Scalable Video Coding (2004? 2007?) (search for "SVC Annex G")

(Keep in mind that software patents aren't about ideas, but specific techniques being used together to achieve an effect, so this isn't intended to be a list of prior art, but rather a list of the same concept being used elsewhere.)

So literally middle out compression - the usual video codec does the bulk of the work, another codec handles uncommon but important cases like sharp text, everything else nobody will notice?

Seems like with LCEVC you still need to decode your encoded frame and compute a difference between the original input to the encoder, at say 30 fps. Many embedded system-on-chips that do H.264/H.265 encoding (eg IP cameras) are not able to decode to memory, only straight to an HDMI/DSI output. Using the CPU to decode and compute a difference is too compute heavy to be possible in real-time on these ARM single or quad core processors.

It looks like they just built something like KrigBilinear, which is open source and patent free. For very pixel-sensitive stuff like colored or anti-aliased text, nothing outperforms KrigBilinar on 4:2:0 sources.

The real fix, obviously, is to just encode as 4:4:4, but MPEG and a lot of popular live streaming video consumers (Nvidia and Steam's in home streaming solutions, Twitch, etc) refuse to accept the truth of the matter.

Google and Bing show zero results for "KrigBilinear". Where can I find out more about this new word?

Because I typo'd. Its KrigBilaterial. Never post on HN before the morning coffee.

https://gist.github.com/igv/a015fc885d5c22e6891820ad89555637

Why are we not seeing more, object recognition and things like camera tracking in video compression?

If you assume that the video you are compressing is made from a 3D scene you should be able to get very high compression ratios.

This is something I've been wishing for for ages - I'm glad to find out it's a real thing now. It's a shame about the licensing, but it's a good start.

How exactly does the specialized upsampler work I wonder?

I always expect video codec announcements to be about neural networks these days.

"Hey guys I've got a great idea, why don't you take your free thing that isn't patent-encumbered (AV1), which you're using SOLELY FOR THAT REASON, and go ahead and wrap that in our patented shit, and pay us?"

Yeah, no thanks.

https://en.wikipedia.org/wiki/LCEVC

LCEVC leverages a base video codec (e.g., AVC, HEVC, VP9, AV1, EVC or VVC) and employs an efficient low-complexity enhancement that adds up to two layers of encoded residuals, along with normative signalled up-sampling methods, that correct artefacts produced by the base video codec and add detail and sharpness for the final output video.

>> "There is Free of Fee License For Integration as Software Player (important for Open Source based OSes like Linux, and Video Players like VideoLan). 𝗕𝘂𝘁 𝘂𝘀𝗶𝗻𝗴 𝗘𝗻𝗵𝗮𝗻𝗰𝗲𝗺𝗲𝗻𝘁 𝗟𝗮𝘆𝗲𝗿 𝗳𝗿𝗼𝗺 𝗲𝗺𝗶𝘁𝗲𝗿 𝘀𝗶𝗱𝗲 𝗶𝘀 𝘀𝘂𝗯𝗷𝗲𝗰𝘁 𝗼𝗳 𝗳𝗲𝗲𝘀."

AV1 likely is patent encumbered and a patent pool already exists for it.

It just hasn't been tested in the courts since its insignificant market share likely doesn't warrant the effort. And Google clearly doesn't have much confidence since they aren't willing to indemnify any users.

Nevertheless, that is the reason for its invention, and for its use, so LCEVC is completely counterproductive if the goal is to not pay for every device sold as a laptop/phone/gpu/cpu/etc maker

Its history is irrelevant.

Right now it likely is patent encumbered and so not really all that different to HEVC only that it has seen minimal adoption amongst hardware/software vendors and content creators and so is infinitely less useful.

Minimal adoption? Don't YouTube and Netflix use AV1 for their highest-quality offerings?

Last I checked that patent pool didn't actually declare which patents they believe are needed for AV1. It's FUD.

Sisvel who operates the patent pool is acting on behalf of companies like Philips, GE, NTT, Ericsson, Dolby and Toshiba.

These are legitimate companies who have a long history of involvement in the codec space and so unless you have some evidence calling it FUD is pretty baseless.

But again if AOM has full confidence that there are no patents issues then they should indemnify users so there is a guarantee it will be royalty-free.

Ah yes, because major companies have never been misleading with patents.

> But again if AOM has full confidence that there are no patents issues then they should indemnify users so there is a guarantee it will be royalty-free.

Name something you're sure about. Would you be willing to bet your entire life savings its true? Most people would say no, regardless of how sure they are.

> Would you be willing to bet your entire life savings its true? Most people would say no

A person's life savings is not comparable to the funds raised by an LLC. An LLC going bankrupt is not the same as a person going bankrupt.

"We have patents required to implement AV1." "Which patents?" "We're not telling. But they're real, take our word for it." Sounds like FUD to me.

Sure, they have a long history of involvement in the codec space ... which makes them even more desperate to stop AV1 by any means, to stop their patent licensing revenue model from going away.

Of course AOM isn't going to open themselves to unlimited liability in the case of some submarine patent. No-one ever does that. E.g. if you buy HEVC patent pool licenses you don't get any such indemnity.

But the nice feature of AOM of course is the Defensive Termination clause in their patent license: https://aomedia.org/license/patent-license/ If one of the companies you listed launches a first-strike lawsuit against any AV1 user, they lose their ability to use AV1.

Until some company decides to put its neck on the block and launch such a lawsuit, or at least identify specific patents they think are necessary, their claims cannot be taken seriously.

> "We have patents required to implement AV1." "Which patents?" "We're not telling. But they're real, take our word for it." Sounds like FUD to me.

Wat. Yeah. I hear "patent pool", I go "okay, just for orientation let me see an itemized breakdown of the fee schedule." That's the logical continuation of that conversation.

In my head, anyway. I'm just a curious observer who wants to understand how much licensors are paying in practice by understanding what they're paying for.

> Last I checked that patent pool didn't actually declare which patents they believe are needed for AV1. It's FUD.

Is it? Is this not the list? https://www.sisvel.com/images/documents/Video-Coding-Platfor...

I stand corrected, my information was a couple of years out of date. Thanks.

AV1 is definately patent 'encumbered' since part of their strategy is MAD style patent counterattacks on anyone who sues over it, withdrawing the use of the patents that are otherwise freely granted to anyone and everyone.

Still leaves an opening for patent troll type operations but pretty much rules out anyone who actually wants to use codecs in their products. Which is why no one has been foolish enough to try to 'prove it' in court, just trying would wreck their own business.

It sounds like they are just saying its a layered approach that can be used with any codec. I'm not a fan of patented codecs, but calling it a grand conspiracy to re-patent av1 is a bit much when really all they are saying is it is independent of codec choice.

I'm not calling it a grand conspiracy, but clearly making money by locking other people and companies into a patented regime is the goal here.

Reply on news.ycombinator.com