Resizing A BTRFS Filesystem

My OEL6 VM ran out of disk space in its root filesystem, so I had to resize the disk. Most of the howto’s are out-of-date or assume if you’re using cryptsetup you’re using LVM or use GParted which doesn’t handle LUKS, or assumes you’re using ext4 not BTRFS.

Actually it was pretty easy once you ignore all of those incorrect tutorials.

1. Resize the VirtualBox VDI file to 40Gb:

VBoxManage modifyhd oracle62.vdi --resize 40000

2. Boot into a Fedora20 LiveCD:

fdisk /dev/sda
delete partition 2 (d)
create new partition 2 (n)
write & quit (w)
reboot

3. When booted back into your VM, resize the BTRFS filesystem to fill you newly grown partition:

btrfs filesystem resize max /
btrfs filesystem defragment /

Full System Backup (and restore)

I thought I’d play around with backups as a continuation of my btrfs experiment. Well basically btrfs can’t do it without dd’ing the entire drive as the UUID’s can’t be changed.

So I thought I’d stick to ext4, and I’ve finally figured out how to do full root filesystem backup including LUKS encryption (without LUKS its easy, you could even do it with tar) using rsync of just the files instead of using dd to backup every bit (17Gb of files instead of 64Gb drive size in my case).

All of the backup section can be done from a live running system.

1. Backup the filesystem UUID’s:

blkid > uuids.txt

2. Backup full MBR (446 boot sector + 64 partition table + 2 sig) and extended partition info:

dd if=/dev/sda of=mbr.img count=1 bs=512
sfdisk -d /dev/sda > sda.sf

3. rsync the root filesystem to a remote server, or external disk, whatever. Obviously only do this to an encrypted filesystem as you’re copying the unencrypted files:

rsync -avp -P --numeric-ids --delete --delete-excluded \
    --exclude-from=excludes.txt / root@ip:/path/

My excludes.txt file looks like this:

# Include
+ /dev/console
+ /dev/initctl
+ /dev/null
+ /dev/zero

# Exclude
- /dev/*
- /proc/*
- /sys/*
- /tmp/*
- /media/*
- lost+found/
- .gvfs/
- .ccache/

That’s the backup done, now to restore to a new disk, do all of this from a live CD, you’ll have to scp the mbr.img and sda.sf files across.

4. Restore the MBR and partition info to a new blank disk:

dd if=mbr.img of=/dev/sda count=1 bs=512
sfdisk /dev/sda < sda.sf

If you want to restore just the boot sector and not the partition table, for example if your disks are different sizes, just change the block count, but you'll have to use fdisk to create the partitions then:

dd if=mbr.img of=/dev/sda count=1 bs=446

5. You'll have to format the filesystems and setup LUKS, but you don't need to create the partitions using fdisk:

mkfs.ext4 /dev/sda1
cryptsetup luksFormat /dev/sda2
cryptsetup luksOpen /dev/sda2 sda2_crypt
mkfs.ext4 /dev/mapper/sda2_crypt

Now for the complicated part. If you just dd the entire disk, then all the partitions would keep the same UUID's, but as we've created new partitions on a blank disk, we have to reset them to the values we captured in #1, this is the part that's not possible on btrfs.

6. Reset LUKS UUID - for this your boot CD will need to have a newer cryptsetup than 1.1.3 found on F14 which doesn't have the ability to change UUID, cryptsetup 1.3.1 as on F16 LiveCD worked here:

cryptsetup luksUUID /dev/sda2 --uuid=12c92874-51ee-11e1-9c56-001d7d00626d

7. Reset root (/) ext4 partition UUID:

tune2fs /dev/mapper/sda2_crypt -U 19086ed4-51ee-11e1-864d-001d7d00626d

8. Reset /boot ext4 partition UUID:

tune2fs /dev/sda1 -U 23162d3a-51ee-11e1-b203-001d7d00626d

9. Finally we need to reinstall grub (I'm not sure why as we used dd on the MBR). So we mount root, and inside that mount /boot, and as we're not backing up /dev, we need to mount the live CD's /dev inside root too, then make that a chroot:

mount /dev/mapper/sda2_crypt /mnt
mount /dev/sda1 /mnt/boot
mount --bind /dev /mnt/dev
chroot /mnt
grub-install /dev/sda

That's it, done. Reboot into your cloned system.

Update: If you have an encrypted swap partition, on first boot you'll also need to run something like:

cryptsetup luksOpen /dev/sda3 sda3_swap
mkswap /dev/mapper/sda3_swap -U 9421cbe2-559f-11e1-9ec6-001d7d00626d

I've just restored my desktop machine into a VirtualBox VM using this method, including shrinking the disk from a 64Gb SSD to a 40Gb virtual disk (as only about 17Gb was used) and it works fine - encrypted swap and root (ext4). Screenshot.

On first boot I made some small modifications to /etc/hosts and /etc/sysconfig/networks to change IP, mac and hostname, and deleted /etc/udev/rules.d/70-persistent-net.rules so it would rescan for eth0 on reboot.

I also ran nvidia-installer --uninstall and deleted /etc/X11/xorg.conf to reconfigure Xorg. Of course if I rsynced the physical machine to the VM, I'd have to redo these steps again, so I've added some files to excludes.txt (such as guest additions).

Btrfs Experiment

After watching this video and reading this blog post, I decided to have a play with the new btrfs (ButterFS) filesystem.

So I downloaded OEL6u2 and made a minimal install in VirtualBox (which doesn’t even include wget or scp!)

I decided not to use LVM, but to use LUKS encryption for my 8Gb ext4 /, a 500Mb ext4 /boot with no encryption, and no swap partition.

Upon first boot I installed the Base Yum repo from here which is essentially just what’s on the DVD, and also the beta repo from here. I then installed the Unbreakable2 kernel and btrfs:

yum -y update kernel-uek
yum -y install btrfs-progs openssh wget
reboot

I then booted from the Fedora 16 LiveCD (not the install DVD) to do the ext4-to-btrfs conversion, as the OEL installer hasn’t yet been updated to include btrfs, although eventually it will allow direct btrfs root installation.

Next I had to decrypt the LUKS partition and do the conversion of the underlying ext4 filesystem:

cryptsetup luksOpen /dev/sda2 luks_root
btrfs-convert /dev/mapper/luks_root

As you need to edit the fstab, make a temporary directory to mount your root filesystem inside root’s $HOME directory:

cd /root
mkdir root
mount /dev/mapper/luks_root root/
vi root/etc/fstab

Simply replace “ext4” with “btrfs” on the “/” line and :wq

For some reason when I tried to boot at this stage I got all sorts of permissions issues and it wouldn’t boot. The fix is to disable SELinux:

vi root/etc/selinux/config
SELINUX=disabled
reboot

Prove we’ve booted into our btrfs root filesystem:

mount | grep btrfs
/dev/mapper/luks-f3b07133-ac92-4d75-a782-d0556e7655dc on / type btrfs (rw)

Remove the old ext4 snapshot and defragment:

btrfs subvolume list /
ID 256 top level 5 path ext2_saved

btrfs subvolume delete /ext2_saved
Delete subvolume '//ext2_saved'

btrfs subvolume list /
btrfs filesystem defragment /

Next I thought I’d try out the the yum plugin that takes a snapshot every time you run yum, so you can rollback (Debian equivalent is apt-btrfs-snapshot):

yum install yum-plugin-fs-snapshot.noarch

Then install Apache to try out the plugin:

yum install httpd
Loaded plugins: fs-snapshot
Setting up Install Process
....
Running Transaction
fs-snapshot: snapshotting /: /yum_20120204121740
....
Complete!

Check for the snapshot, note its ID and reboot:

btrfs subvol list /
ID 256 top level 5 path yum_20120204121740

reboot

To boot into the snapshot made before you installed Apache, press “e” at grub prompt to edit the parameters and insert the following into the kernel line:

rootflags=subvolid=256

Then press “b” to boot. Once booted prove Apache isn’t installed anymore, and reboot back into the default snapshot:

rpm -qa |grep httpd
reboot

Check apache is installed again:

rpm -qa |grep httpd
httpd-tools-2.2.15-15.0.1.el6.x86_64
httpd-2.2.15-15.0.1.el6.x86_64

Magic! So no we’ve converted an ext4 filesystem to btrfs on top of a LUKS partition, and have proven yum snapshots work and can boot into a snapshot without needing any backup/restore system.

The snapshot is just regular files, so could be tarred up and moved to an external drive for backup I guess:

ls /yum_20120204121740/
bin   dev  home  lib64       media  opt   root  selinux  sys  usr
boot  etc  lib   lost+found  mnt    proc  sbin  srv      tmp  var

I tried to simulate a catastrophic drive failure – i.e. if you’ve got a tarball of a snapshot, can you restore that to a blank disk with empty ext4 /boot and btrfs / partitions? Well the answer is no if you’re using LUKS. By using dd to backup the MBR and /boot partition I got somewhere, however when I restored the snapshot (untarred it into an empty btrfs filesystem) it booted without prompting for the LUKS key, and then died.

So it still seems a bit like btrfs is for rolling back to a previous point, or to be used with one or more drives in a RAID array, but no use when a single drive fails.

Update: I found that because btrfs stores the BLKID of the disk in every data block, you could never actually move a btrfs filesystem to another disk! The only way to do it (as I have) is to dd the entire disk to another disk, and then they’re BLKID’s will be the same, also it solves the problem of trying to piece together the /boot sector and grub in the MBR.

The problem however, is that as you can’t mount two devices with the same BLKID’s at the same time, you couldn’t use rsync to sync the two disks. So for backup of btrfs, you’re stuck with dd’ing the entire disk, or don’t use LUKS and use hardware RAID (not btrfs RAID or software RAID) instead, which is pretty crap.