It works a lot like rsnapshot (each backup is a self contained snapshot of a system at that point in time, and only the incremental changes are transferred to the backup server), but with compression and full file level deduplication (not only across backup generations, but across all files including ones from multiple clients). It uses GNU `find` and `tar` on the client to do the heavy lifting of getting the files (so only a simple shell script is needed on the client, and then only if you are doing push backups). On the backend side, it stores the metadata in an SQLite table, and individual files are stored in a vault directory using lzop compatible compression.
The advantages over tools like rsnapshot is that it has compression, full file level deduplication, and metadata is stored in a DB file (which avoids a large hardlink-farm for dedup (avoiding problems like https://news.ycombinator.com/item?id=8305283), also removes the need for root privileges on the backup server to store file ownership/permissions). The disadvantages compared to other tools, is it doesn't do encryption on the client (this would break dedup if done right, and I'd rather leave encryption to the experts), and it only does file level dedup, so isn't suitable for backing up VM image files (unless you combine it with something like libguestfs, something I'll document in an upcoming release).
There are also some nice-to-have features, for example if you have a job to expire all daily backups older than 10 days, and monthly older than 6 months -- and you have a host that hasn't been running / backed up for a year, the expire job will keep a minimum number of backups for each host (default 3 backups, but is tuneable). And the next major release will support multiple repositories, so you can move old backup sets to archival storage (external USB disks, possibly tape library support, etc). Also included is support for storing extended / SElinux attributes, and the current client shell script has a plugin architecture to execute pre/post backup steps for app-specific backups, such as Oracle hot backups.
Edit: for Windows backups, I'm currently running it under Cygwin. But I'm not able to preserve Windows specific permissions. To fix this, I need a "tar" implementation for Windows which stores the extended ACL information in PAX headers in the tar file, similar to how Red Hat's patched GNU tar does SElinux attributes. If I can't find something, I may take a crack at it myself.
EDIT: it will require some manual fiddling, especially if have multiple machines, so it's not an option that passes all criterias from the article.
For cloud-based backups, my preference would be to backup locally first, then sync that to a cloud service, since restoring from a local box is faster than remote. It might be a fun project to create an intermediary server which the tarsnap client talks to, and have that server then forward everything to the cloud service.
It's just for the Tarsnap service.
And since the client isn't open source (but is distributed as source code), would it be a license violation if someone reverse-engineered the protocol to write an open server component?
It's not clear: https://www.eff.org/issues/coders/reverse-engineering-faq
For personal/development use on Linux, you should absolutely take a look at obnam (and the similar attic and bup). They are simple software that do dedup, encryption and cloud storage.
I do mainly Linux stuff, and obnam looks nice and feature-packed enough to fit my current needs, I'll dig in to that a bit. Cheers :-)
Of course, this is very Linux centric, but it fits my needs and likewise allows me to rsync back anything on the backup at any time, even if I accidentally all of /bin/ or something equally stupid. Though now I've somewhat updated it (I use a fully encrypted system and set cryptsetup to use pass-files on the main disk for the backups in addition to a set password), but the fundamentals are all there.
[1] https://thatgeoguy.ca/blog/2013/12/26/encrypted-backups-in-d...