> intra-archive-file redundancy
What software do you use for this?
What software do you use for this?
On Gentoo Linux, the name of the software package that provides them is "app-arch/par2cmdline".
"par2create" needs 2 parameters, the amount of redundancy to be added and the name of the input file (which should be a compressed then encrypted archive file).
It produces several additional files, with recovery information. My script creates then a directory, moves there all the files, including the original archive file, and then aggregates the directory with all its files into a single ".pax" archive file. The ".pax" file can then be stored on a couple of HDDs or of tape cartridges.
Before creating the initial archive file, I scan the directories to be archived and I gather all the metadata of the files (names, sizes, timestamps, content hashes), which I store in a database. The metadata can later be used to search for files, to find out from which archival device they should be restored, and also to detect duplicates (by using the content hashes) that may be omitted from later backups.
When retrieving such a file with redundancy, after extracting its component files the "par2verify" utility verifies the integrity and reconstructs corrupted data, if necessary.
For working with tape drives, I prefer to use a FreeBSD server, because the FreeBSD utilities for tape drives and in general for SAS/SCSI peripherals are more convenient and more complete than those of Linux (it is also possible to use a FreeBSD VM on a Linux computer, with PCIe pass-through for the SAS HBA card used to attach the tape drive).
It is inefficient to store small files on tapes, so I usually collect data to be archived on tapes until I reach an amount of about 50 GB, then I compress, encrypt, add redundancy, aggregate again into a single file and then save it into the tape cartridges.