telendram

AES-NI can be used as a hash, but don't assume that it makes a good job at it. Latest version of MeowHash is fine, but all other variants and prior versions have shitty collision ratios, equivalent to a ~50 bits hash : https://github.com/Cyan4973/xxHash/wiki/Collision-ratio-comp...

Even falkhash is not that great, and feature an abnormal amount of collisions as the nb of hashes increase. Basically, all "naive" AES implementations share this design weakness.

Without `--ultra`, the decompression memory budget is capped at 8 MB. At `--ultra -20`, it's increased to 32 MB.

That's still less than XZ, which reaches 64 MB.

0.8% is with Arch's default settings. It's fairly strong, but not the strongest, to preserve cpu during compression.

zstd is used at level 20, but it can compress more. Levels can go up to 22, and (complex) advanced commands are available to compress data even more.

A little known fact is that parallel XZ do compress worse than XZ ! I measured pixz as being approximately ~2% worse than xz. That's because input is split into independent chunks.

In comparison, the 0.8% of zstd looks like a bargain.

Beyond more profound changes to the underlying transport protocol, bandwidth acceleration can be provide through a combination with fast compression.

A good setup here would be to pipe data to `tar`, then `zstd`, then `ssh`/`scp`, and do the reverse on the receiver side.

From an end-user standpoint, all this could be abstracted away if one could get `wget` (named in the article) to support zstd compression natively, which is possible since zstd is also a web compression standard.

Such compression support is, by the way, available in `wget2`.

FP languages tend to consider correctness first, but treat performance, and predictability of performance, as second grade citizens. By performance, I mean both time and space, aka cpu and memory. Cpu is bad, but Memory is by far more critical.

This makes it tricky to correctly anticipate the outcome of a bunch of lines of code. One also needs to worry about the compiler, how it will interpret that, and this can be hard. Worse, the compiler's interpretation can change with a different version, or following a tiny source code modification that looked harmless. No one likes wild performance swings for no good reasons.

Add to that picture layers of abstraction that make debugging an executable more difficult, lack of choice and capabilities in said debugging tools (generally justified by "my code never has any bug!"), and it makes developing a headache, at least when things start to go awry. I haven't even started with 3rd party library supports, etc.

On top of that, one rarely codes in the void : a source code is going to be shared with a "community", so it's not a solo choice. One cannot impose a rarely used FP language to a team that know another language very well, and has never asked to change that.

Don't get me wrong, I love FP languages, and I can even see how some of their best concepts are being "borrowed" (ha!) by more declarative languages. That may be the best outcome. Maybe in the future, the differences will be less pronounced, the same way as RISC and CISC architecture have largely merged nowadays to inherit the best parts of both sides.

That's a huge stretch. Integer arithmetic is perfectly portable. Defined operations have bit-exact defined results, reproducible on any platform.

In contrast, the result of basic floating point operations is not. You may live in a world were you use a single cpu type, and a single OS, and a single compiler, in which case, it might appear to you as defined. But anyone involved in some portability across multiple systems know how improbable it is to rely on floating point for anything exact. Actually, don't. ever.

For anything which requires exact results, use integer instead.

I'm fine with "just" `struct as strong type` part ...