> Those use a rather expensive hash function (you really want to avoid hash collisions), [...]
Then we are clearly not thinking of the same kind of software.
> but (at least some ten years ago) memory, not processing speed, was the limiting factor.
In what I described, IO is the limiting factor. You want to avoid having to read the whole file, if you can.
I think you are thinking of block level online deduplicators that are integrated into the file system?