Full System Backup (and restore)

I thought I’d play around with backups as a continuation of my btrfs experiment. Well basically btrfs can’t do it without dd’ing the entire drive as the UUID’s can’t be changed.

So I thought I’d stick to ext4, and I’ve finally figured out how to do full root filesystem backup including LUKS encryption (without LUKS its easy, you could even do it with tar) using rsync of just the files instead of using dd to backup every bit (17Gb of files instead of 64Gb drive size in my case).

All of the backup section can be done from a live running system.

1. Backup the filesystem UUID’s:

blkid > uuids.txt

2. Backup full MBR (446 boot sector + 64 partition table + 2 sig) and extended partition info:

dd if=/dev/sda of=mbr.img count=1 bs=512
sfdisk -d /dev/sda > sda.sf

3. rsync the root filesystem to a remote server, or external disk, whatever. Obviously only do this to an encrypted filesystem as you’re copying the unencrypted files:

rsync -avp -P --numeric-ids --delete --delete-excluded \
    --exclude-from=excludes.txt / root@ip:/path/

My excludes.txt file looks like this:

# Include
+ /dev/console
+ /dev/initctl
+ /dev/null
+ /dev/zero

# Exclude
- /dev/*
- /proc/*
- /sys/*
- /tmp/*
- /media/*
- lost+found/
- .gvfs/
- .ccache/

That’s the backup done, now to restore to a new disk, do all of this from a live CD, you’ll have to scp the mbr.img and sda.sf files across.

4. Restore the MBR and partition info to a new blank disk:

dd if=mbr.img of=/dev/sda count=1 bs=512
sfdisk /dev/sda < sda.sf

If you want to restore just the boot sector and not the partition table, for example if your disks are different sizes, just change the block count, but you'll have to use fdisk to create the partitions then:

dd if=mbr.img of=/dev/sda count=1 bs=446

5. You'll have to format the filesystems and setup LUKS, but you don't need to create the partitions using fdisk:

mkfs.ext4 /dev/sda1
cryptsetup luksFormat /dev/sda2
cryptsetup luksOpen /dev/sda2 sda2_crypt
mkfs.ext4 /dev/mapper/sda2_crypt

Now for the complicated part. If you just dd the entire disk, then all the partitions would keep the same UUID's, but as we've created new partitions on a blank disk, we have to reset them to the values we captured in #1, this is the part that's not possible on btrfs.

6. Reset LUKS UUID - for this your boot CD will need to have a newer cryptsetup than 1.1.3 found on F14 which doesn't have the ability to change UUID, cryptsetup 1.3.1 as on F16 LiveCD worked here:

cryptsetup luksUUID /dev/sda2 --uuid=12c92874-51ee-11e1-9c56-001d7d00626d

7. Reset root (/) ext4 partition UUID:

tune2fs /dev/mapper/sda2_crypt -U 19086ed4-51ee-11e1-864d-001d7d00626d

8. Reset /boot ext4 partition UUID:

tune2fs /dev/sda1 -U 23162d3a-51ee-11e1-b203-001d7d00626d

9. Finally we need to reinstall grub (I'm not sure why as we used dd on the MBR). So we mount root, and inside that mount /boot, and as we're not backing up /dev, we need to mount the live CD's /dev inside root too, then make that a chroot:

mount /dev/mapper/sda2_crypt /mnt
mount /dev/sda1 /mnt/boot
mount --bind /dev /mnt/dev
chroot /mnt
grub-install /dev/sda

That's it, done. Reboot into your cloned system.

Update: If you have an encrypted swap partition, on first boot you'll also need to run something like:

cryptsetup luksOpen /dev/sda3 sda3_swap
mkswap /dev/mapper/sda3_swap -U 9421cbe2-559f-11e1-9ec6-001d7d00626d

I've just restored my desktop machine into a VirtualBox VM using this method, including shrinking the disk from a 64Gb SSD to a 40Gb virtual disk (as only about 17Gb was used) and it works fine - encrypted swap and root (ext4). Screenshot.

On first boot I made some small modifications to /etc/hosts and /etc/sysconfig/networks to change IP, mac and hostname, and deleted /etc/udev/rules.d/70-persistent-net.rules so it would rescan for eth0 on reboot.

I also ran nvidia-installer --uninstall and deleted /etc/X11/xorg.conf to reconfigure Xorg. Of course if I rsynced the physical machine to the VM, I'd have to redo these steps again, so I've added some files to excludes.txt (such as guest additions).

Back Online!

I got fed up with my shitty web host and their downtime – apparently they fucked up the OpenVZ kernel when running “yum update” on their CentOS server, then had RAID controller issues, then their ISP null-routed them after they got blacklisted by Spamhaus, then they fucked up the config of their new host node, blah, blah, lots of excuses (including flooding in Australia depsite the servers being in Amsterdam/Munich/Kansas, and poor outsourcing).

Anyway, I’ve moved to a UK company now and Email and websites are back online.

Luckily I had backups from a couple of days ago when the old server was briefly online, so I setup a new server and rsync’ed to it, then from that I rsync’ed to a 2nd server, and rsync’ed the differences from home. That meant server1 was online overnight and server2 was online a few hours later.

Funnily enough I’ve got a better resource/price deal than with the old hosts!

101 NASL’s

I’ve just finished writing some new Nessus plugins, taking my NASL count to over 100 now.

Just as I finished checking them into Git, Tenable decided to renumber the plugin ranges. Custom NASL’s were always given a range around 50000-53000, but now Tenable are up to 50321 themselves, so have decided on a new set of ranges:

Passive: 1 – 10,000
Active: 10,001 – 900,000
Custom: 900,001 – 999,999
Compliance: 1,000,000+

I’ve made some changes to my backup regime too, from now on I’m backing up my whole $HOME directory using BackInTime to an encrypted drive, rather than encrypting a tarball. This saves space as BIT uses rsync and hard links to create incremental backups. The old tar+gpg method would create a 3Gb file per backup, with BIT I’ve got 11 incremental backups totalling 9Gb.

Decrypting, decompressing and unpacking a 3.5Gb tarball to get to perhaps one file inside it is painfully slow, with BIT I can instantly restore (or just view or copy) a file at any date.

As it uses rsync as a backend its also simple to run from cron, which you can’t really do with GnuPG as you need to enter your passphrase.

I was thinking of using Deja Dup as its nicely integrated into Nautilus in Fedora/Ubuntu but its GUI is pretty minimal – literally a button or menu item for backup/restore/revert, and I’m not keen on the backend or limited use of GnuPG (passwords not keys, and no password input checking).

A Doh! Moment

I just went to rsync my desktop in the UK with the backup drive from France and got the command backwards, so ended up with the old UK files overwriting the new France files on the backup drive!

Luckily I correctly rsync’ed my $HOME directory first, so its only downloads and backups that got nuked. I can live without them for a week. My VM’s and work/office stuff are fine.

Note to self: next time proof-read the command before running it as root, and mount the backup drive readonly!

Xmas 2009

I had a lovely Christmas, spent most of the time over at M&D’s eating and drinking too much! I think they’re coming over for New Year’s Eve. We all went out for Chinese on Xmas Eve and I went over to PP’s afterwards.

I got lots of presents including a Senseo coffee machine which I have in my computer room as well as an electric toothbrush, clock radios, toiletries and booze.

I just watched Dorian Gray, which is right up there with Watchmen as the worst comicbook movie ever.

As I’d like to upgrade my PC to Fedora 12, but still need some F10-only applications (NessusClient 4.0.2 for example) I decided to clone it into a Virtual Machine.

I made a basic Fedora 10 64-Bit install under VirtualBox, then rsync’ed the filesystem using variations on the below commands (as root) making sure not to copy over /proc, /sys, /dev, /tmp and so on:

rsync -ap --delete --numeric-ids /sbin/ 192.168.0.133:/sbin/
rsync -ap --delete --numeric-ids /var/ 192.168.0.133:/var/
rsync -ap --delete --numeric-ids /home/ 192.168.0.133:/home/
rsync -ap --delete --numeric-ids /bin/ 192.168.0.133:/bin/
rsync -ap --delete --numeric-ids /lib/ 192.168.0.133:/lib/
....

I modified grub/fstab (screwed up a bit there and had to use a RHEL rescue CD) and ran mkinitrd to remove the encryption, LVM and Vista partition; then replaced the Nvidia drivers with VirtualBox ones, disabled Compiz/screensaver etc. It now works very well, I’ve shrunk the memory down to 1Gb and only 2 cores and the disk only uses about 17Gb. It quite amazing what you can do using rsync on a live system, who needs cloning programs like Acronis/Ghost?

Its the ultimate backup as not only can I simply re-run the rsync commands to keep it up-to-date, but I have a completely running system not just my data, and can move it to any machine I like. I could probably even migrate it back to a physical machine if I needed too.