Ask HN: Is data on an encrypted file-system harder to recover when corrupted? | Hacker News Reader

Ask HN: Is data on an encrypted file-system harder to recover when corrupted?

5 pointswebmobdev6 comments

I would like to use the encryption feature of my file systems (APFS, HFS+, ZFS) but have this irrational (perhaps ignorant?) fear that in case of some minor software or HDD / SSD hardware failure (e.g. bad sectors), more data loss may occur because of the encryption and it would also be harder to recover data from an encrypted file system. Is this fear rational? (All my file systems are currently unencrypted, and I do take regular backups.)

View on HN

My two cents.

It is very rational.

It may depend on the exact method of encryption, but encryption (full disk or filesystem) - generally speaking - makes recovery impossible (as opposed to harder), at least for the affected files.

The good (or bad) news is that SSD's tend to fail anyway in an unrecoverable manner[0], hdd's had more (though still scarce) probability of recovery.

And it is needed to mention how most if not all file formats (besides plain text and a few other non-compressed formats) are anyway (even if not encrypted) rather hard to recover.

Unless you have some (very relevant) needs for encryption, avoid using it, you are anyway adding a layer of complexity to all operations on disk.

[0] in some cases the SSD firmware will protect itself by making the SSD "read only", but this happens normally after the controller has already found corruption, while recovering a partially corrupted filesystem is usually doable, recovering a partially corrupted encrypted filesystem is often impossible.

Like you rightly pointed, the disadvantage of using encryption on full-disk or file system seems quite high. But I don't get it - if a file system is not robust enough to handle data corruption and aid recovery when encryption is enabled, the file system would earn a bad rep. So I am assuming that that they would factor that and make the encryption system equally robust?

> ... makes recovery impossible (as opposed to harder), at least for the affected files...

This is my main doubt in an encrypted file-system - is encryption done on the individuals files on it or is the whole file-system encrypted (implying the whole partition is one giant encrypted blob)? The latter would imply corruption in any part of the file system would make it next to impossible to recover any file in it. If each individual files are encrypted separately (on the fly), then any corruption would only affect some files, and the rest of the files would still be recoverable.

There are mainly two "levels" or "approaches" to recovery, file system based and file based.

Let's put aside for a moment encryption.

What you want to recover (normally) is the contents of a disk, i.e. the files on it, a file in itself, if contiguous[0] is written to an "extent" on disk, i.e. it starts at a given sector and ends on another given sector, in the case of a fragmented file the file is scattered over several extents.

The file system structure are essentially an index or address table to these extents, you look for the file in it and you get its address.

If these structures are corrupted, you can still recover easily the contiguous files by "carving" the disk, the process for fragmented files is difficult, lengthy, prone to errors and usually works only for a fraction of the files.

With encrypted files this carving is simply impossible.

About encryption as said it depends on the exact type of encryption, full disk encryption (hardware or software) encrypts everything, file system encryption usually also encrypts the file system structures, but it could also not, as an example NTFS EFS:

https://en.wikipedia.org/wiki/Encrypting_File_System

encrypts the files but not the structures (your "encryption done on the individual files"), while (say) FreeOTFE or Truecrypt/Veracrypt:

https://en.wikipedia.org/wiki/VeraCrypt

encrypts as well "everything" (your " the whole partition is one giant encrypted blob").

It depends.

There is also hardware full disk encryption (as an example there are USB disks that include encryption) that is even worse as the decryption depends on the specific hardware controller in the USB disk case, which may (or may not) be tied to the connected disk without a (known) way to re-couple the disk to an identical controller, so that if the controller fails the data is lost.

https://en.wikipedia.org/wiki/Hardware-based_full_disk_encry...

But still, if you have a non-encrypted JPEG file, changing as little as 1 (one) single byte in it, you can often make it invalid/unviewable anymore, loosely any kind of compressed file may already be harder or impossible to recover.

[0] this is the single most important feature for file based recovery, unlesss the extent(s) where the file resides are unreadable you can recover fully the file, losing only the file system metadata (filename, date, etc.)

Full disk encryption is generally not exactly one giant encrypted blob where any modified bit means the blob is bad and can't be decrypted. If it were, it would be impossible to modify.

Instead, it's broken up into a large number of small blobs. If your disk encryption has a checksum, a bit flip on the storage results in a checksum failure (hopefully) and a read error; it might be difficult to read sectors in that blob, but you could read the rest. Not all disk encryption systems use checksums, so maybe you just get a bitflip in the read data. Of course, if the bitflip is in the keystorage, that may not be recoverable at all; many systems allow you to export the keys to allow for recovery in this case, and they may store multiple copies in any case.

The data is encrypted per small chunks. Corruption in a chunk renders that chunk useless, but usually won’t impact other chunks. There is small sensitive data, for example the header of a LUKS volume containing the password and encryption details, that if corrupted, the whole disk can be lost. You can back up this header.

So, no, a small corruption in an encrypted disk or FS won’t necessarily impact the entire system.

In ZFS, a bit flip corrupts files in which it occurs, not the whole pool (with or without encryption).

On the other hand, you are right to be worried that encryption is another layer that can go wrong. As mentioned, password and encryption details must not be corrupted.

> You can back up this header.

Is this particular to any one FS or applicable for all production FS (NTFS, APFS, HFS+, ZFS etc.).

Reply on news.ycombinator.com