Hacker News Reader

Comment by JoshTriplett | Hacker News Reader

> My zlib implementation for example is consistently 25% faster than the reference version, despite me simply "hand compiling" it straight from the C source.

Have you compared it to Intel's optimized implementation of zlib, at https://github.com/jtkukunas/zlib/ ?

If you can improve on that implementation, please consider submitting a patch.

View on HN

I am aware of Intel's patch re: psubusw usage, and interestingly chose the same solution when hand-compiling it as theirs well before I saw their patch to it (in fill window, which does make a substantial difference).

Have you compared my code to [your?] repo yet? I will endeavour to do so, though I am not sure submitting a fasm-based patch to that repo would make sense. Cheers

I hadn't compared the code yet; I figured it would make sense to compare performance first.

Congrats on your work 2ton_jeff!

Regarding zlib, is that the fastest implementation that's currently available?

I rememeber stumbling into a guy that claimed his implementation was way faster (2x or more) than the original zlib, and it was a drop-in replacement and scaled fairly well. Unfortunately, I can't find it right now (it should be bookmarked on another computer).

Various people have attempted to speed up zlib; it's not that high a bar, if you're OK with not producing output binary-identical to the original zlib. Deflate compression algorithms keep references to possible LZ backreferences in a hash table, and depending on your choice of hash algorithm, and how much you prune your table versus spending memory and time storing and searching it, you'll end up emitting different backreferences. The implementation at https://github.com/jtkukunas/zlib/ replaces the hash algorithm with something much faster.

libslz is really, really fast (and fairly new). It's used in the next version of haproxy. In my own tests it's proved substantially faster than both zlib and miniz.

Reply on news.ycombinator.com