Hacker News Reader

Comment by snuxoll | Hacker News Reader

For long-term archival I think relying on your compression software to protect data integrity is a fool's errand, protecting against bit-rot should be a function of your storage layer as long as you have control over it (in contrast to say, Usenet, where multiple providers have copies of data and you can't trust them to not lose part of it - hence the inclusion of .par files for everything under alt.binaries).

View on HN

I keep seeing recommendations for par/par2 but it seems like as software, the project isn't actively maintained? As an aside, that makes me think of dead languages and the use of latin for scientific names because it isn't changing anymore... but do you want that out of archival formats and software?

There's a par2 "fork" under active development - https://github.com/Parchive/par2cmdline

The fork compiled for me this week, when the official 0.3 version on Sourceforge wouldn't. I vaguely remembered par3 being discussed, but couldn't find anything usable. And that's an example of why to be wary of new formats, I guess?

It's probable that PAR2 is essentially feature complete, so no maintenance is really needed.

The program does pretty much the same thing as it did a decade ago.

Yes. Learn from what we have already experienced in the world of computing.

* https://www.theguardian.com/uk/2002/mar/03/research.elearnin...

Nope. You always need end-to-end parity integrity checking. Your data goes through too many layers before reaching the storage medium. E.g. I once got a substantial amount of my pictures filled with bit errors because of a faulty RAM module in my NAS.

This happened to me, and caused me to rethink my approach file management.

Unfortunately, mainstream tooling is largely fire-and-forget and never includes verification (e.g. copying succeeds even if the written data is getting garbled), so one is forced to use multi-step workflows to get around this. It's pretty discouraging that no strong abstractions exist in this space.

Yes, end-to-end checking is a must - but that applies to any method of integrity protection. I could run TrueNAS at home on some old desktop I've retired instead of the used Dell R520 I bought for the task, but I have experienced memory failures before and expect them to happen - this doesn't change if you're using .par files instead.

(People underestimate how frequently memory corruption can actually occur, almost two years ago when Overwatch first came out the game kept crashing - it took me forever to find the cause was a faulty DIMM. Hell, right now the R320 I have in my rack at home has an error indicator because one of my 2 year old Crucial RDIMM's has an excessive amount of correctable errors).

Reply on news.ycombinator.com