Btrfs Experiment

After watching this video and reading this blog post, I decided to have a play with the new btrfs (ButterFS) filesystem.

So I downloaded OEL6u2 and made a minimal install in VirtualBox (which doesn’t even include wget or scp!)

I decided not to use LVM, but to use LUKS encryption for my 8Gb ext4 /, a 500Mb ext4 /boot with no encryption, and no swap partition.

Upon first boot I installed the Base Yum repo from here which is essentially just what’s on the DVD, and also the beta repo from here. I then installed the Unbreakable2 kernel and btrfs:

yum -y update kernel-uek
yum -y install btrfs-progs openssh wget
reboot

I then booted from the Fedora 16 LiveCD (not the install DVD) to do the ext4-to-btrfs conversion, as the OEL installer hasn’t yet been updated to include btrfs, although eventually it will allow direct btrfs root installation.

Next I had to decrypt the LUKS partition and do the conversion of the underlying ext4 filesystem:

cryptsetup luksOpen /dev/sda2 luks_root
btrfs-convert /dev/mapper/luks_root

As you need to edit the fstab, make a temporary directory to mount your root filesystem inside root’s $HOME directory:

cd /root
mkdir root
mount /dev/mapper/luks_root root/
vi root/etc/fstab

Simply replace “ext4” with “btrfs” on the “/” line and :wq

For some reason when I tried to boot at this stage I got all sorts of permissions issues and it wouldn’t boot. The fix is to disable SELinux:

vi root/etc/selinux/config
SELINUX=disabled
reboot

Prove we’ve booted into our btrfs root filesystem:

mount | grep btrfs
/dev/mapper/luks-f3b07133-ac92-4d75-a782-d0556e7655dc on / type btrfs (rw)

Remove the old ext4 snapshot and defragment:

btrfs subvolume list /
ID 256 top level 5 path ext2_saved

btrfs subvolume delete /ext2_saved
Delete subvolume '//ext2_saved'

btrfs subvolume list /
btrfs filesystem defragment /

Next I thought I’d try out the the yum plugin that takes a snapshot every time you run yum, so you can rollback (Debian equivalent is apt-btrfs-snapshot):

yum install yum-plugin-fs-snapshot.noarch

Then install Apache to try out the plugin:

yum install httpd
Loaded plugins: fs-snapshot
Setting up Install Process
....
Running Transaction
fs-snapshot: snapshotting /: /yum_20120204121740
....
Complete!

Check for the snapshot, note its ID and reboot:

btrfs subvol list /
ID 256 top level 5 path yum_20120204121740

reboot

To boot into the snapshot made before you installed Apache, press “e” at grub prompt to edit the parameters and insert the following into the kernel line:

rootflags=subvolid=256

Then press “b” to boot. Once booted prove Apache isn’t installed anymore, and reboot back into the default snapshot:

rpm -qa |grep httpd
reboot

Check apache is installed again:

rpm -qa |grep httpd
httpd-tools-2.2.15-15.0.1.el6.x86_64
httpd-2.2.15-15.0.1.el6.x86_64

Magic! So no we’ve converted an ext4 filesystem to btrfs on top of a LUKS partition, and have proven yum snapshots work and can boot into a snapshot without needing any backup/restore system.

The snapshot is just regular files, so could be tarred up and moved to an external drive for backup I guess:

ls /yum_20120204121740/
bin   dev  home  lib64       media  opt   root  selinux  sys  usr
boot  etc  lib   lost+found  mnt    proc  sbin  srv      tmp  var

I tried to simulate a catastrophic drive failure – i.e. if you’ve got a tarball of a snapshot, can you restore that to a blank disk with empty ext4 /boot and btrfs / partitions? Well the answer is no if you’re using LUKS. By using dd to backup the MBR and /boot partition I got somewhere, however when I restored the snapshot (untarred it into an empty btrfs filesystem) it booted without prompting for the LUKS key, and then died.

So it still seems a bit like btrfs is for rolling back to a previous point, or to be used with one or more drives in a RAID array, but no use when a single drive fails.

Update: I found that because btrfs stores the BLKID of the disk in every data block, you could never actually move a btrfs filesystem to another disk! The only way to do it (as I have) is to dd the entire disk to another disk, and then they’re BLKID’s will be the same, also it solves the problem of trying to piece together the /boot sector and grub in the MBR.

The problem however, is that as you can’t mount two devices with the same BLKID’s at the same time, you couldn’t use rsync to sync the two disks. So for backup of btrfs, you’re stuck with dd’ing the entire disk, or don’t use LUKS and use hardware RAID (not btrfs RAID or software RAID) instead, which is pretty crap.

Fecking Fedora

I made the mistake of trying the Yum Upgrade method of moving my rock-solid F13 install to F14 before they end-of-life’d it. Well that resulted in a machine that would constantly reboot at the “Loading operating system….” bit just before Grub. I couldn’t fix it no matter how many times I tried variations of “grub-install /dev/sda”.

So I decided I might as well bite the bullet and do a fresh install of Fedora 15. The first problem there was that my computer becomes very unreliable when I plug in the DVD drive (I really have to buy a new PSU) so I created a bootable USB installation using the UNetBootIn instructions. Well that took literally hours to write 3Gb to my flash drive on Ubuntu for some reason and then wouldn’t boot anyway, the fix was to type “/isolinux/vmlinuz initrd=/isolinux/initrd.img” at the boot: prompt.

So I merrily went and selected a smallish installation, not every package a I usually would do. Apparently this came to over 2000 packages! 4 hours later its still installing, then I notice the switch is blinking like crazy, guess what – despite unticking the “Use Fedora Updates repository” option, it seems to be downloading 2000+ RPM’s from the internet!

Anyway, eventually I’m booted and into Gnome3 – well what a screwup that is – I can’t even find where to configure the screensaver! The only thing I like about it is I don’t have to spend a day configuring compiz-fusion plugins as most of what I use is built-in and preconfigured now (like drag’n’drop windows between workspaces). Gnome-Shell I hate, what is it with having to search through a list of all your applications rather than just selecting from a menu?!

I found this site which at least tells you how to move the stupid clock and get the shutdown menu back again.

The SysV-init replacement “systemd” doesn’t integrate with any of the GUI tools for configuring services and doesn’t work with akmods so every kernel update is going to be fun with NVidia/VirualBox. I can see that becoming the next PulseAudio heathen.

For some reason NFS is all screwed up with Nautilus and won’t let a regular user unmount (although it will mount!) despite what /etc/fstab says. So I went back to autofs, which at least these days seems to cope with the remote server being offline. Then I noticed that USB drives won’t mount – turns out I can’t use /media for NFS mounts anymore, so have to create a specific /nfs directory now:

/etc/auto.master:

/nfs /etc/auto.nfs --timeout=3600 --ghost

/etc/auto.nfs:

data0 -fstype=nfs4,hard,intr,timeo=10,rw,rsize=32768,wsize=32768,noatime server:/data0
data1 -fstype=nfs4,hard,intr,timeo=10,rw,rsize=32768,wsize=32768,noatime server:/data1

So I’m pretty much back at a working system now, although I’m getting the odd hard-lockup like I was with F14 (despite removing USB/DVD devices) so I think I need a new PSU. Odd that F13 was fine.

Skype works, Firefox4 by default, Nessus’ F13 RPM works, VirtualBox has a new F15 RPM, I can rsync all my servers etc. I rebuilt my RPM’s for JtR, gSTM, gIP and knockd and I built VirtualBox4 and CyanogenMod7 using Sun JDK.

I noticed that even if you run “ssh-add” you still get a popup dialog box from gnome-seahorse-whatever asking for the passphrase. Apparently ssh-add is deprecated, and ssh-agent is fired up by the dialog now, so just enter the passphrase in there and it will cache it in the agent.

Nothing but Totem will play .FLV files for some reason – I think RPMFusion broke their MPlayer build or something.

Whenever I fire up QGit now I get some annoying warning:

An error occurred while executing command: git repo-config --get i18n.commitencoding
Git says: WARNING: git repo-config is deprecated in favor of git config.

One thing I noticed was that when restoring, BackInTime (root) sets all the directory modification times to now, although files get their original datestamp. Bit of a shame that, as it means sorting a directory listing my date is pretty ineffectual.

Firefox 4

Fedora 13/14 only have Firefox 3.6 in their repositories, so I decided to install the “Spot” repo and try out the new Firefox 4.0:

su -
cd /etc/yum.repos.d/
wget http://repos.fedorapeople.org/repos/spot/firefox4/fedora-firefox4.repo
yum install firefox4

You can also get it from the Remi repo.

It seems to have copied a lot of the GUI changes from Google Chrome – you now have the “Firefox Button” which replaces the menubar for all preferences etc; just like Chrome’s spanner icon, tabs have been moved to the top of the window, and the statusbar is gone.

The bookmarks toolbar has been replaced by a button, which is handy for netbooks with small screens, but on my 24″ 1920×1200 LCD I put the toolbar back on – you can configure it back again like all the new features, so if you want you can make 4.0 look like 3.6

It seems a bit faster than 3.6, although the quoted 5x I’d say is crap. Tab grouping looks handy as does app tabs.

Update: urgh, it seems for the sake of the Mac fanbois, they’ve removed folder icons from the bookmarks menus – think of sub-directories inside the personal toolbar.

To fix this create ~/.mozilla/firefox/<profile>/chrome/userChrome.css with the following content:

menu.bookmark-item > .menu-iconic-left {
  visibility: visible;
}

You can also set this as a Gnome-wide option using GConf:

gconftool-2 -s --type=bool /desktop/gnome/interface/menus_have_icons true

Or if you’d prefer to stick to not displaying the icons but want to remove the whitespace from the menu where the icon should go, use this:

menu.bookmark-item > .menu-iconic-left {
  display: none;
}

Or if you are a Mac fanboy and want to go the whole hog and remove folder icons even from the personal toolbar use this instead:

toolbarbutton.bookmark-item > .toolbarbutton-icon {
  display: none;
}

Fedora Rebuild

I finally gave up fighting with the crashing kernel issue in F14 so went back to F13. For some reason when I booted my restored Clonezilla image the PC would just reset, so I did a fresh F13 install from DVD.

I was suprised actually that I had 90% of my settings and software setup and installed within about 3 hours. It used to take a whole day to customise the desktop and install all those weird packages. I guess a lot of those packages are now in the standard Fedora install, and the rest I’ve made my own RPM’s for (john, gip, gstm, knockd etc.) or can be got from the RPMFusion, Google and Adobe repositories.

Google don’t seem to host a .repo file, but they get installed if you manually install the RPM’s for Chrome/Earth but you can create them manually in /etc/yum.repos.d/

google-chrome.repo

[google-chrome]
name=google-chrome
baseurl=http://dl.google.com/linux/chrome/rpm/stable/x86_64
enabled=1
gpgcheck=1

google-earth.repo

[google-earth]
name=google-earth
baseurl=http://dl.google.com/linux/earth/rpm/stable/x86_64
enabled=1
gpgcheck=1

And then import the key from https://dl-ssl.google.com/linux/linux_signing_key.pub

Essentially my post-install transaction is:

rpm -ivh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-stable.noarch.rpm
rpm -ivh http://download1.rpmfusion.org/nonfree/fedora/rpmfusion-nonfree-release-stable.noarch.rpm

rpm -ivh http://linuxdownload.adobe.com/adobe-release/adobe-release-i386-1.0-1.noarch.rpm
rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-adobe-linux

rpm -Uvh http://www.linux-ati-drivers.homecall.co.uk/flashplayer.x86_64/flash-release-1-2.noarch.rpm

yum remove mc emacs gnome-games

yum -y install yum-plugin-fastestmirror xmms xmms-mp3 xmms-faad2 xmms-pulse xmms-skins \
audacious audacious-plugins-freeworld* \
rhythmbox gstreamer-plugins-ugly gstreamer-plugins-bad gstreamer-ffmpeg \
amarok xine-lib-extras-freeworld \
mplayer mplayer-gui gecko-mediaplayer mencoder gnome-mplayer ffmpeg \
xine xine-lib-extras xine-lib-extras-freeworld gstreamer-plugins-ugly \
gstreamer-plugins-bad gstreamer-ffmpeg vlc gnochm chmviewer comical \
AdobeReader_enu dkms akmod-nvidia  xorg-x11-drv-nvidia-libs.i686 \
flash-plugin google-chrome-unstable comix backintime-common backintime-gnome \
screenruler xinetd gftp nvclock* compiz-fusion-extras seahorse seahorse-plugins \
nautilus-open-terminal compiz-fusion-unsupported-gnome compiz-fusion-unsupported \
compiz-fusion-extras-gnome compiz-gnome compiz-fusion compizconfig-backend-gconf \
compiz-manager compiz-fusion-extras conky lm_sensors gkrellm kernel-devel kernel-headers \
GConf2-gtk GConf2 thunderbird-lightning qgit urbanterror amap gloobus-preview.x86_64 \
alsa-lib-1.0.23-1.fc13.i686 libXScrnSaver-1.2.0-1.fc12.i686 qt-4.6.3-10.fc13.i686 \
qt-x11-4.6.3-10.fc13.i686 ccsm alacarte VirtualBox-4.0.x86_64 amap urbanterror ethtool \
purple-plugin_pack.x86_64  purple-facebookchat.x86_64 purple-plugin_pack-pidgin.x86_64 \
pidgin-sipe.x86_64 pidgin-otr.x86_64 pidgin-guifications.x86_64 pidgin.x86_64 \
binutils gcc make patch libgomp glibc-headers glibc-devel kernel-headers \
kernel-devel dev86 iasl qt4-devel pulseaudio-libs-devel glibc-devel.i686 \
libgcc.i686 texlive-texmf-latex java-1.6.0-openjdk-devel zlib-static glibc-static \
libstdc++.i686 libxslt-devel libIDL-devel SDL-devel libXmu-devel libstdc++-static \
wget google-earth-stable.x86_64 unrar bcrypt tsclient perl-Spreadsheet-WriteExcel \
expect freetype-freeworld gconf-editor p7zip fbreader-gtk redhat-lsb.i686 \
gnome-media-apps

yum -y update

Plus installing the VirtualBox, Nessus, nmap, msttcore-fonts RPM’s I already have downloaded.

I don’t bother with Oracle’s VirtualBox repository as its not updated enough so its quicker to download the RPM’s manually from their website (you have to download the extpack anyway).

Rawhiiiiiiiiide!

No, not the cowboy series, but the Fedora development repository.

As part of my investigation into why large transfers are hard locking my PC, I was advised to install the kernel from Rawhide, which also meant enabling the RPMFusion Rawhide repository to pull in the Nvidia modules. So first we install the repo:

yum install fedora-release-rawhide.noarch

Then we edit the repo files to enable them but limit them to kernel/nvidia RPM’s (we don’t want to upgrade to Fedora 15 Alpha!) and not debuginfo/source:

/etc/yum.repos.d/fedora-rawhide.repo

[rawhide]
name=Fedora - Rawhide - Developmental packages for the next Fedora release
failovermethod=priority
#baseurl=http://download.fedoraproject.org/pub/fedora/linux/development/$basearch/os/
mirrorlist=https://mirrors.fedoraproject.org/metalink?repo=rawhide&arch=$basearch
enabled=1
gpgcheck=0
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch
includepkgs=kernel*

/etc/yum.repos.d/rpmfusion-nonfree-rawhide.repo

[rpmfusion-nonfree-rawhide]
name=RPM Fusion for Fedora Rawhide - Nonfree
#baseurl=http://download1.rpmfusion.org/nonfree/fedora/development/$basearch/os/
mirrorlist=http://mirrors.rpmfusion.org/mirrorlist?repo=nonfree-fedora-rawhide&arch=$basearch
enabled=1
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmfusion-nonfree-fedora-latest-$basearch file:///etc/pki/rpm-gpg/RPM-GPG-KEY-rpmfusion-nonfree-fedora-rawhide-$basearch
includepkgs=*nvidia*

Then running “yum update” pulled in the 2.6.38-0.rc7.git2.3.fc16.x86_64 kernel from Fedora 16 (and Nvidia modules) and it does seem to have fixed the issue – I did a full rsync backup and a couple of 1.3Gb NFS transfers to test.

VirtualBox works fine too thankfully – dkms/akmod rebuilt the kernel modules upon reboot and my CentOS 5.5/Ubuntu 9.10/OEL6 guest VM’s work fine.

Mum’s thinking of getting an Amazon Kindle3 for her birthday, and it seems they now allow reading of regular PDF/epub files not just Amazon’s DRM-protected Mobi format, and there’s a Linux program called Calibre that enables you to convert between the ebook formats, including .cbz comic books.

Update: the new kernel doesn’t seem to have fixed the issue, I’ve had several crashes today without much network traffic. Starting to wonder if its an Nvidia driver issue or a hardware issue like HDD/PSU/RAM. I ordered a new PCIe NIC with Via Velocity VT6130 chipset instead of Realtek RTL8111D but doubt its going to fix anything now 🙁

I’ve also been playing with the Kindle and Kobo Android apps and downloading a shedload of free ebooks from Amazon to read on my phone – even got Calibre to convert useless PDF’s to more portable epub files. The Kobo website is useless as its throws DRM-protected PDF’s at you (for free books!) but the Android app just downloads unencrypted epubs.