Bzip2 is slow. That’s the main issue. Gzip is good enough and much faster. Also, the fact that you cannot get a valid bzip2 file by cat-ing 2 compressed files is not a deal breaker, but it is annoying.
Yes, I do. Zstd is my preferred solution nowadays. But gzip is not going anywhere as a fallback because there is a surprisingly high number of computers without a working libzstd.
Indeed for many simple test cases zstd is both better and faster.
Despite that, it is possible to find input files for which zstd is either worse or slower than gzip, for any zstd options.
I have tested zstd a lot, but eventually I have chosen to use lrzip+gzip for certain very large files (tens of GB file size) with which I am working, because zstd was always either worse or slower. (For those files, at the same compression ratio and on a 6-year old PC, lrzip+gzip has a compression speed always greater than 100 MB/s, while zstd only of 30 to 40 MB/s.)
There are also other applications, where I do use zstd, but no compressing program is better than all the others in ALL applications.
https://cbloomrants.blogspot.com/2021/03/faster-inverse-bwt....
TIL. Now that's why gzip has a file header! But, tar.gz compresses even better, that's probably why it hasn't caught on.
It’s handy because it is very easy to just append stuff at the end of a compressed file without having to uncompress-append-recompress. It is a bit niche but I have a couple of use cases where it makes everything simpler.
that being said, speed is important for compression so for systems like webservers etc its an easy sell ofc. very strong point (and smarter implementation in programs) for gzip
Like a software installation that you do one time. I'd not even want it for updates if I need to update large data files. The only purpose I'd see is the first-time install where users are okay waiting a while, and small code patches that are distributed to a lot of people
(Or indeed if size is important, but then again bzip2 only shines if it's text-like. I don't really find this niche knowledge for a few % optimization worth teaching tbh. Better teach general principles so people can find a fitting solution if they ever find themselves under specific constraints like OP)
Long comment to just say: ‘I have no idea about what I’m writing about’
These compression algorithms do not have anything to do with filesystem structure. Anyway the reason you can’t cat together parts of bzip2 but you can with zstd (and gzip) is because zstd does everything in frames and everything in those frames can be decompressed separately (so you can seek and decompress parts). Bzip2 doesn’t do that.
So like, another place bzip2 sucks ass is working with large archives because you need to seek the entire archive before you can decompress it and it makes situations without parity data way more likely to cause dataloss of the whole archive. Really, don’t use it unless you have a super specific use case and know the tradeoffs, for the average person it was great when we would spend the time compressing to save the time sending over dialup.
I think their argument is sound and it makes using bzip2 less useful in certain situations. I was once saved in resolving a problem we had when I figured out that concatening gzipped files just works out of the box. If not, it would have meant a bit more code, lots of additional testing, etc.
how to compress data has little to no relation to how this compression can be implemented in programs. How its implemented, will reflect on how the quality of the algorithm is perceived, becaus e the two are not seperate from a user perspective.
This isn't accurate.
1) Most zstd streams consist of a single frame. The compressor only creates multiple frames if specifically directed to do so.
2) bzip2 blocks, by contrast, are fully independent - by default, the compressor works on 900 kB blocks of input, and each one is stored with no interdependencies between blocks. (However, software support for seeking within the archive is practically nonexistent.)
https://github.com/facebook/zstd?tab=readme-ov-file#benchmar...
And, yes, I prefer LZMA over the obsolete Bzip2 any day, but GZIP it's like the ZIP of free formats modulo packaging, which it's the job of TAR.
That should not happen too often, considering that IIRC bzip lasted only a couple of months before being replaced by bzip2.