Hacker News Reader

Comment by devy | Hacker News Reader

Yeah, I had the same thought. Also gzip has been around for many decades and newer lossless compression algorithms are way better both in terms of speed (comparing with the same hardware) and compression ratio, e.g. LZ4 algorithm and Zstandard.

It just doesn't make sense for the HW engineers and chip designers to optimize backwards for older software algorithm instead of optimizing forward for newer algorithms.

View on HN

This depends on the prevalence on the different algorithms. If let's say gzip is 90% of the decompression load then it would make good sense.

One such extremely common workload that does frequent gzip compression is HTTP servers because it is the standard method for compression.

GZip is a widely supported compression type for HTTP connections. It makes complete sense to have it be hardware accelerated.

Zstandard is not actually that fast!

I recently benchmarked it versus the ancient "DEFLATE" algorithm, as seen in the .NET Framework.

They're virtually identical in speed using typical settings.

Zstandard only has a performance advantage if using the prepared dictionaries, in which case it is genuinely faster.

If you don't go to the effort of "training" a dictionary and using it, there's zero benefit to Zstandard.

IBM tech is for large old heavy-footed enterprises. I expect much GZIP, less Zstandard there.

How specific are those "gzip instructions", though? Are there newer algorithms that could benefit from some of the same acceleration primitives, even if they do higher-level things differently?

They're not instructions; it's a dedicated hardware unit. Most of the area appears to be devoted to finding matches so it should be possible to add other LZ-style algorithms without much additional area.

Reply on news.ycombinator.com