You can complain about philosophies but this is just using the wrong tool for the job. Complain about bzip if you feel the bzip authors should have made multithreaded implementation for you.
You can complain about philosophies but this is just using the wrong tool for the job. Complain about bzip if you feel the bzip authors should have made multithreaded implementation for you.
The old approach was that programs don't need internal parallelism because you can get it by just piping stuff and relying on the kernel's buffering to keep multiple processes busy.
Eg, tar is running on one core dealing with the filesystem, gzip is running on another core compressing stuff.
In the early days, Windows would have a single program doing everything (eg, Winzip) on a single core, while Unix would have a pipeline with the task split into multiple programs that would allow for this implicit parallelism and perform noticeably better.
Today this is all old and doesn't cut it anymore on 128 core setups.
Other algorithms like zstd and gz resulted in much lower compression rates.
I'm sure there is a more efficient solution, but changing three letters in a script was pretty much the maximum amount of effort I was going to put in.
On an unrelated note, has someone already made a meta-compression algorithm which simply picks the best performing compression algorithm for each input?
Do you know if there are any tests showing which compressor is better (compression wise) for which data?