A generally safe backup process looks like this:
Production has no access to backup.
Backup has read only access to production.
Backup writes are append and not overwrites.
Deletes/archival are governed by a retention process.
Production has no access to backup.
Backup has read only access to production.
Backup writes are append and not overwrites.
Deletes/archival are governed by a retention process.
1. Gain access to change the code on the front-end web servers (usually PHP)
2. Change the database access layer to transparently encrypt data being written to the database, and decrypt data being read from the database. The key would be loaded into memory by curl'ing an attacker-controlled website at startup.
3. Wait 30 days
4. Notify the company that they're compromised, turn off the attacker-controlled key service, and restart the web front end
Now step (3) ensures that most data in the database has been re-written, and if your backups are dumps of the production database, you now have a month of encrypted backups that you can't read... If you're lucky, you may have a month-old backup to restore from; if you're unlucky, you rotate every 30 days.
Implementing zero downtime transparent encryption on any moderately complex codebase above the database layer sounds like a big engineering challenge. Even moreso if you're trying to do it without the sysadmin noticing. Even moreso if there is no persistence for the encryption key so every user request ends up having to fire a request to the attackers server - that'll kill performance. I doubt many attackers would attempt it.
Read through any modern multi stage remote exploit. The amount of engineering required to pull these stunts off - reliably no less - is truly staggering, and yet attackers continue to innovate and develop new techniques and even more complex exploits.
And you overestimate the competency of the defenders: sysadmin? What small business has a dedicated sysadmin? It’s not like we are talking about running a global hosted chat service like slack here. (Oh wait...)
BTW Persistence for the key in memory is trivial: create a shared memory segment and store the key there, with a ttl.
Was it a site in maintenance mode, where no one was working on anymore?
Oh, and there must not be a DBA at the company either, since absolutely nobody ran a SQL or analytics query in an entire month and noticed that some of the rows were full of unreadable garbage..
This story must be about someone's single server, single dev LAMP blog from ten years ago if it ever actually happened.
Yes, we are talking about the long tail of little LAMP servers handling small business needs with some custom code written by a contracting shop. What’s a devops? DBA? Once the site is developed and working, there’s little reason to pay to change it.
Edit : thinking about it, this story doesn’t add up. Doesn’t this mean the client must have a copy of the decryption key, meaning any cached client would render the ransom demand worthless? It’s just also so much easier, if someone has that level of access, to make a copy of the database, encrypt it, then blow away the old one. Doing a silent deploy of client code with no one noticing seems way harder.
Honestly, it used to be Eastern Europe, until we started realizing they've got some serious programming talent and contracted some work out. Now it's less bad, though still a bit sketchy. Of course, this doesn't help nation-state attacks.
Where should I send the resume?
Where it breaks down is assuming only the web server that is compromised has access to the database.
Of course, it's also possible it's not, and I wonder if that's more likely. Every industry has such stories; maybe this is the white whale of forensics.
It's nice to have a restore step in there too, so one has both a validation that the backup is usable, and gives a "playground" where one can have a safe day old environment for testing/training/whatnot against production data.
That is essential. If you do not regularly test restoring, you do not know if you have backups. I normally overstate this just a little for effect, "If you don't test your backups you don't have backups", but that isn't precisely true.
And yes, it frequently makes sense to base test/fixture data and whatnot on backup data flows, but unless you don't deal with any PII, you're hopefully not using that data without a sanitization step.
To anyone viewing my answer as a guide: a backup by itself is not a continuity plan. There are other important areas such as setting recovery metrics (RPO/RTO) and testing restores against those metrics. The world is rife with stories of unreadable backups and/or restorations that took days instead of hours.
The network is segregated to limit impact if they do get hacked.
To keep everyone familiar, Data Recovery processes are practiced regularly.
...