Hacker News Reader

Comment by jbk | Hacker News Reader

(President of VideoLAN here)

What you should really look at on dav1d is the fact that the code has now around 186 kLoC of hand-written assembly in .S and .asm files…

I think this is quite a feat (this is more asm than the whole FFmpeg) and this is very rare those days to write so much asm.

View on HN

I hope there's someone fuzzing all those variants well to find vulnerabilities, that's a lot of hand written code parsing untrusted data.

It is tested by Google's OSS-Fuzz among others: https://github.com/google/oss-fuzz/tree/master/projects/dav1...

One of the bugs found by OSS-Fuzz: https://bugs.chromium.org/p/oss-fuzz/issues/detail?id=11464

I can't see any configuration in that repo for testing different asm variants.

There's cpu = 'i686' in the config file which would in fact seem to imply none of the modern archs mentioned in a different subthread here ("x86_64(SSE3, AVX2), ARMv7 and ARMv8") are fuzzed.

> 186 kLoC of hand-written assembly

That's for x86_64(SSE3, AVX2), ARMv7 and ARMv8 right? Is the 186kLoC equally distributed between those arch or is there one arch that has most of the optimization focus?

Is it all hand-written, or do you have some kind of script/macros to create a good part of this assembly (like OpenSSL does IIRC)?

All arch have the same level of optimizations, but AVX2 might have a couple functions more. AVX-512 is still behind…

Hand written with some macros, notably for the x86 mess (windows calling convention, ssse being 32 and 64b).

pretty damning state of current compilers, I would conclude. I thought that they got much better in the meantime.

but having a look at clang or GCC, I can confirm. still baby steps, years ahead for proper optims.

Reading this it sounds like you are discouraging using it due to the implementation being big and hard to maintain. Did you mean it this way?

I really can't see how you read anything else that "it's very optimized" in my message, tbh.

"[..] code has now around 186 kLoC of hand-written assembly in .S and .asm files…"

This reads to me as it being a PITA to maintain. Cross platform code is usually a pain, cross platform with assembly optimizations is more of a pain. Optimization nearly always makes things harder to maintain, and this sounds like it was optimized to hell and back...

I have a suspicion that the president of VideoLAN has a better idea of the complexity & maintainability of this code than you do.

> This reads to me as it being a PITA to maintain.

Not more than any other language: when it's well done, it's manageable. When it's spaghetti code, it's not manageable.

No. Most encoders/decoders need assembly to get last drop of performance. Most of the times it comes down to when you give up. The team here was quite relentless to build a usable AV1 decoder. A few years ago this was a pipe dream and jbk and team have done a tremendous job.

In addition to other noting that decoders and encoders almost all have an enormous amount of hand optimized code because they are literally the most demanding applications most people run (which is why most codecs today are hardware accelerated and they are used in almost all CPU benchmarks) 186k LOC (given that it's assembler which is nearly always "vertical code") is also relatively small for decoders.

People don't really get how complex video codecs are.

It's not they're writing a library full of assembly for the sake of it. Reading the readme, one can discover they have dedicated optimised implementations for a large range of processors, which in turns means there's a lot of assembly involved

Reply on news.ycombinator.com