Can you survive your Data Hiroshima?

Even expensive and top-of-the-line hardware is fallible. Last night (at the time of writing) my main workstation’s PSU burned. I mean, not soft-failed and powered down, I mean burned. With the acrid smell filling the room, I knew something went very wrong the instant I entered my study. I found my computer powered down, non-responsive. I wasn’t too worried because I knew that even if the computer went dead for good, I would not loose much data since, you know, I have backups.


Are you capable of surviving your own little Data Hiroshima?

So I after a few quick tests, I gather that the computer is dead for good. I don’t know the extend of the damage, but by the smell, it doesn’t look good. Inspecting the computer with a flashlight (well, yeah, maybe I watched too many CSI episodes) I gather that the motherboard and other components are not damaged—no burn marks and no exploded capacitors. So I remove the drives and get to a local computer shop to get a new power supply unit to be installed.

Apparently, light fixtures are very expensive in Nevada

Apparently, decent light fixtures are very expensive in Nevada

When I got the computer back, I ran an extensive memory test to make sure the RAM survived. I then had the S.M.A.R.T. hard drives to self-test using smartctl tools (and they’re already in the Ubuntu repositories). Apparently, the computer survived the failure with no detectable problems. However, these tests take a good while (several hours) so I disassembled the faulty PSU to get a good look at it in the mean time.

The PSU burned indeed (shown opened on my workbench)

The PSU burned indeed (shown opened on my workbench)

Not a very nice sight.

Years ago, when I was still using Windows, I would have been very unnerved by this event because 1) you cannot easily transfer a Windows OS from one computer to the other by simply exchanging drives and 2) I wasn’t too hot on redundancy and backups.

The first problem is largely solved by Ubuntu. Exchanging a drive from a machine to the other of same general architecture (e.g., from AMD64 to AMD64) results in very little problems: change the Ethernet adapter number from eth0 to eth1 and possibly change the graphics drivers (using sudo dpkg-reconfigure xserver-xorg). Otherwise, your OS will reboot just fine. Unlike Windows which will probably BSOD if drivers are missing or fail to initialize. I have no idea what a hard drive transplant does to the Windows Genuine Advantage software, but I’m sure you won’t like it.

The second problem is solved by my systematic backing up of my data. You may also wonder right now just how reliable your data backup strategy is. Let us consider a few:

  • Copies of files on the same disk. Somewhat protects you from bad manipulations as the file can be recovered from a different location on the same disk. If the disk fails, you loose everything.
  • Copies of files on different disk(s), same computer. Provides some protection against disk failure. RAID arrays (with many different levels) offer you protection against individual drive failure by using redundancy spread across many drives in the same machine. If your machine is heavily damaged, you may lose all data. For example; a bad PSU that shorts everything, including sufficiently many drives to circumvent the RAID redundancy.
  • Copies on a different machine, on-site. Periodical synchronisation of files against a another file-system protects you from catastrophic failure of a single machine, whether or not you have many drives and redundancy in the same machine, but does not protect you against a site-wide catastrophic event, however unlikely, such as, well, I don’t know, flash flooding or fire.
  • Copies on a remote media or machine off site. Periodical synchronisation of files against a removable media (something like, say, a USB hard drive that you keep off-site or that you bring from a secure location to exchange with the current copy. So you make a full (or incremental) backup one media then go to the secure location and exchange it for the second media, which you bring back for the next backup. In this way, you minimize the time both copies are at the same location. So, even if the site is nuked from orbit, the other copy should survive in the secure location.

You do realize that none of those techniques are absolutely bad and that they work better when you combine them. You may keep working copies in two locations on your file system (maybe using a revision control system), which is protected by RAID, which is periodically synchronized with a remote, but one-site computer for fast retrieval, which is in turn periodically backuped on the off-site medias should something really bad happen.

Oh noes! I had no backups!

Oh noes! I had no backups!

I must confess that I do most of the above. I do not have machines configured with RAID drives, but I do sync them nightly on a dedicated backup machine. I do make copies on removable drives. I do exchange them once in a while from a secure location. You may think it’s stupid, that I cannot possibly have data that’s worth that much trouble. Well, of course, I beg to differ. Not only do I have lots of source code, test data, and music, I also have a number of documents that I cannot get back from other sources. The LaTeX source code from my Ph. d. thesis and many other texts. The 30 000 or so pictures I shot over a period of several years. And countless other little things.

* *

As our lives become increasingly dependent on technology—whether it’s our online presence on social networking sites or our use of digital photography to record our memories—it makes plenty of sense to make sure that none of this information is lost forever because of a faulty computer.

So, start training for your backup jutsu right now.

7 Responses to Can you survive your Data Hiroshima?

  1. Mathieu Pagé says:

    Hi, it’s funny you posted this yesterday because without reading it yet I started thinking about and testing a way to backup my pictures.

    I already have on-site backup on a differents machine for things like source code and other documents. I periodically (every two weeks) copy theses backup on a couple of DVD that I bring to work and store in my desk drawer.

    However, this is not really suitable to store 20gb of photos (.jpg + raw format, about 25Mb per picture), it would require 5 DVDs, for now. My collection is growing rapidly.

    So I examined two options: 1) Two external copies on hard drives. Given the cost of hard drives theses days, this is not too costly. I could still store the hard drives at work. 2) Online backups on Amazon S3. This is the solution I’m currently testing. The main advantage of this technique in my opinion is that once it’s setted up you can forget it. Every time I’ll upload pictures from my camera to my computer they’ll get uploaded to S3 the following morning. I’ll also set it up so that my nightly backup will be uploaded, maybe once a week of every two weeks. This is not too costly either. S3 storage cost 0.15$ per Gb/month + 0.10$ per GB uploaded. So it should cost me 2$ to upload my data, then 3$ per month therafter. So it depends on how much data you want to backup. If you want to backup 500 Gb, it may be more wise to go with the hard drives technique, but for something under, maybe, 100 Gb, S3 might be a viable option.

  2. How do you manage your data’s privacy? Are you using some kind of encryption to prevent your stuff from being “leaked”? This is a real concern since it has happened before that websites like this got tons of data downloaded through a security breach of some sort.

  3. mathmoi says:

    For now, I don’t. I’m only uploading photos as a test and it wouldn’t be that bad if they were leaked in any way. However, that’s definitely something I need to think about before I upload my regular backup that include all my other stuffs like source code and /home of all my machines.

  4. I’m asking because I’m curious about how people do this generally. I suppose that the vast majority just “trusts” the services (like people “trust” facebook to sell keep their private information private)

  5. mathmoi says:

    Well, I’m definitely not trusting Amazon with my data, but maybe it is because we’re programmers and we know how easy/frequent it is company inadvertently leaks data. I can easily imagine most peoples simply trusting Amazon/Google/Facebook.

    I’m not sure how I will do it, but I guess I can add a tool like crypt (not look at it yet) to my backup script to encrypt the files before I upload them.

  6. […] I mentioned before (here and here), you really can’t trust your hardware to maintain a good health all by itself. It can […]

  7. […] I’ve said before, having a good personal data backup strategy is important and it turns out that even small things […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: