Hacker News Reader

Comment by jfoutz | Hacker News Reader

backups are for suckers. keep the data on a few different spinning disks. if you can solve data synch between two sites, just keep your data synched.

it's much better to ask yourself how long to replicate your existing system then how to back up. pxe boot to a kernel that you can install over the network with, bcfg2 to get the thing up to spec, start copying data.

a lot of machines can be back and configured in 5 minutes.

that said, i'm not you. i don't have terabytes of data to do statistics on. maybe there are other horrible details i'm forgetting. fast rebuilding is a pretty awesome strategy for a lot of cases.

View on HN

maybe there are other horrible details i'm forgetting

Yes.

Why should I bother to write this? I'll outsource the task to the authors of High Performance MySQL, Second Edition, page 475:

Backup Myth #1: "I Use Replication As a Backup"

This is a mistake we see quite often. A replication slave is not a backup. Neither is a RAID array. To see why, consider this: will they help you get back all your data if you accidentally execute DROP DATABASE on your production database? RAID and replication don't pass even this simple test. Not only are they not backups, they're not a substitute for backups. Nothing but backups fill the need for backups.

Obligatory xkcd link: "Bobby Tables" http://xkcd.com/327/

Completely reasonable response, forgive me for being an inarticulate noob in my post.

I thought delayed replication was one of the main strategies they advocated in that book. I don't have it on hand. my mistake.

This is the incremental backup script I use on my Linux box at home, a quick-and-dirty imitation of what Time Machine does. Obviously $HOME/backup is a different physical disk. Feel free to improve on this.

--------

#!/bin/bash

HOME=

date=`date "+%Y-%m-%dT%H:%M:%S"`

rsync -aP --link-dest=$HOME/backup/current /home

$HOME/backup/back-$date

rsync -aP --link-dest=$HOME/backup/current /etc $HOME/backup/back-$date

rm $HOME/backup/current

ln -s back-$date $HOME/backup/current

#see if the disk is getting full

FREE=`df -lk|grep sdb1|awk -F" " '{print $5}'|awk -F"%" '{print $1}'`

#alert me if the backup disk is getting full.

T=80

if [ "$FREE" -gt "$T" ]

then

    df -lk| mail $myaddress -s"disk alert $T% capacity"

else

    echo "backup disk is less than $T full"

You should think about keeping an offsite backup if possible. A second disk won't necessarily help if your computer gets dropped in a pool, house catches on fire, etc.

that is very interesting - I though replication was the nearest to realtime "backup" as one could get. Is there any other way, one can do incremental, realtime backups for, say, mysql/postgres ?

Replication doesn't defend you against deleting the wrong file, or messing up an update to your database, or any of a large class of PEBKAC issues.

It's also not useful if your main files get corrupted and you diligently propagate the corruption. See: ma.gnolia

Setting aside the database, file problems are fairly easily solved via svn, .snapshot or a good configuration engine.

As far as the database goes, I'm big on stored procs + archive tables, but i'll leave that to the grown ups ;)

...until you forget to check in important changes, or neglect to add a seemingly trivial configuration file to your SCM.

Whereas, a policy of automatically backing up everything except for your exclude list would have saved your bacon.

What if you hit rm -rf * and that gets synched?

I don't mean to be rude, I don't know anything about you, but if you were a system administrator working for me, today would be your last day.

Of course it would be his last day, he's a sysadmin and he just typed rm -rf * on /

You know, you tell yourself that you understand rm -rf. And yet somehow you always find a way to type it on something you shouldn't. Computers are perverse objects.

It's like reading an airline crash report, which I'm told often ends up sounding like a comedy of errors. Most airline crashes have an entire handful of causes, all of which are individually innocent, but on the one extremely rare occasion when they all happen at the same time they add up to disaster.

I fat-fingered the destination directory in a script once and accidentally shrunk my entire photo archive to 128x128 thumbnails. Thankfully I did have backups of all but a handful.

I'm always amazed at the number of people who spend their time thinking about replication strategies without understanding that data can be accidentally deleted in production too. I guess backups look "easy", so it's not as sexy an area of architecture planning.

People just don't always grasp the big picture. People that see RAID as a backup strategy are only looking at 'hard drive failure' as potential scenario. The same for machine replication. They are only trying to prevent against what happens if 'the machine dies.' As long as they realize that this doesn't protect against, "I accidentally deleted the files," then it's all good.

In a previous job I had a so-called system administrator laugh at my paranoia when I suggested that his awesome backup system -- protecting the equivalent of several dozen person-years of scientific data, millions of dollars' worth -- wouldn't protect us against a fire in the building. Or, for that matter, a thief who liked to steal computer hardware.

Job security note for sysadmins: When someone suggests a disaster scenario, don't open your response with a laugh.

Well, you know that you're talking to someone that doesn't plan for the future when, "What happens if the building burns down?" is responded to with, "You're just being paranoid." I guess the appropriate response would be to point out that using his logic tons of taxpayer money could be saved by nixing the fire departments (and tons of corporate money could be saved by not paying for fire protection -- alarms, detectors, escapes, etc).

if you can solve data synch between two sites, just keep your data synched.

LOL! And if you get a corrupt block on your primary site, what're you going to do? All your standbys are instantly tainted!

Better leave this one to the grownups.

Replication is not backup. It only protects against hardware failure.

Spinning disks are good though. A lovely spot for backups. Just put them in a different building.

Edit: In the time it took me to write this 5 other people also lambasted this poor fellow. Ouch.

It doesn't matter if your data is replicated across 3 different continents. If all of your data is accessible behind the same admin login, or if it is all vulnerable to the corruption in the original, then it still has only 1 point of failure. A proper "spinning" backup is an immutable journaling system that can't be trimmed except via extremely secure methods (e.g. physical access to the machine).

A real-time mirror won't help if your data gets corrupted (via a fat-fingered shell command, errant script, or a compromised system). When it's appropriate, it's awesome, but I don't think it's a replacement for a static "offline" backup,

Sorry, this is not only WRONG but actually HARMFUL advice.

Redundancy is not a backup. If someone with full admin control to your system can destroy all of your data then you do not have backups. A proper backup is physically separate from your primary data and, preferably, can't be destroyed with mere admin access to the system. The number of sites that have had catastrophic data loss due to relying on mirroring instead of true backups is quite significant.

Reply on news.ycombinator.com