Skip to content

HOWTO install Raspbian to a Native ZFS Root Filesystem, or, How I Learned to Love Data Integrity

Chris edited this page Jan 13, 2020 · 3 revisions

This is a walkthrough for installing zfs from debian packages on a Raspberry Pi running Raspbian.

Get the OS onto a card with room to spare for zfs, or you can use/boot a USB drive with only /boot on the SD Card

  • Write OS to sd card, edit config.txt. Remove init=/usr/lib/raspi-config/init_resize.sh
  • create “ssh” file in boot partition to enable ssh

Boot the rpi

uncomment deb-src line in sources.list and update packages

sudo vi /etc/apt/sources.list

sudo apt-get update ; sudo apt-get -y dist-upgrade

Remove unused kernel to make room on /boot and / (We will be very tight on space with a fresh, unexpanded install)

sudo rm /boot/kernel.img

sudo rm -rf /lib/modules/4.14.30+

Install build tools and depends.

sudo apt-get -y install build-essential fakeroot devscripts autogen libelf-dev debian-keyring busybox initramfs-tools

sudo apt-get clean

sudo apt-get -y build-dep zfs-dkms spl-dkms

mkdir ~/src ; cd ~/src

dget -x http://http.debian.net/debian/pool/main/s/spl-linux/spl-linux_*.dsc

dget -x http://http.debian.net/debian/pool/contrib/z/zfs-linux/zfs-linux_*.dsc

curl "https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=826994;filename=sysvinit_patch_5_826994.diff;msg=112" > init.patch

patch zfs_arc_max off by 1

curl https://github.com/zfsonlinux/zfs/commit/7403d0743e2b75b7f5412a14007ba159efb67a7d.patch > ob1.patch

Build it

cd zfs-linux-0.7.6 ; cat ../init.patch ../ob1.patch | patch -p1 ; cd ..

cd spl-linux-0.7.6 ; debuild -i -us -uc -b --no-lintian ; cd ..

cd zfs-linux-0.7.6 ; dpkg-source --commit ; debuild -i -us -uc -b --no-lintian ; cd ..

sudo apt-get install raspberrypi-kernel-headers

sudo apt-get clean

sudo dpkg -i spl_*_armhf.deb spl-dkms_*_all.deb

disable swap (you can also rm the zfs and spl build directories to make more room)

sudo swap off -a ; sudo rm /var/swap

sudo dpkg -i libnvpair1linux_*_armhf.deb libuutil1linux_*_armhf.deb libzfs2linux_*_armhf.deb libzpool2linux_*_armhf.deb zfs-dkms_*_all.deb zfsutils-linux_*_armhf.deb zfs-zed_*_armhf.deb zfs-initramfs_*.deb

Agree to the zfs license warning, then pick “No” to not abort building on a 32-bit kernel.

reboot

Check to see if zfs is loaded, if not load it.

lsmod | grep zfs

modprobe zfs

create a zfs pool on the SD Card (USB drives would be sda):

sudo fdisk /dev/mmcblk0

p (print part table, look when p2 ends (3629055 on mine)

Device Boot Start End Sectors Size Id Type

/dev/mmcblk0p1 8192 93802 85611 41.8M c W95 FAT32 (LBA)

/dev/mmcblk0p2 98304 3629055 3530752 1.7G 83 Linux

n (new)

p (primary)

3

First sector is end of p2+1 = 3629056

end is default or +xG to make a x GB partition

w (to write partition table, likely need to reboot to reread the partition table)

Now create a pool, set the temp mount root to /mnt/rastank, and create a root dataset for easy snapshotting and rollbacks. It’s always a good idea to never put files in the pool without a dataset.

sudo zpool create -o ashift=12 -O canmount=off -O compression=lz4 -O normalization=formD -O mountpoint=/mnt/rastank -O xattr=sa -O relatime=on -O recordsize=1M -R /mnt/rastank rastank /dev/mmcblk0p3

sudo zfs create -o mountpoint=/ rastank/raspbian-stretch

Now you can copy your system to the new ZFS partition. Use tar to preserve hard links and any special files. Install pv and di as helpful status utilities. If you don't want the progress to show, pipe the first tar command to the other.

sudo apt-get install pv di dialog

(cd /; sudo tar cf - --one-file-system . ) | pv -p -bs $( sudo du -sxm --apparent-size / | cut -f1 )m | (sudo tar -xp /mnt/rastank)

If you want it even more pretty, use this command and it uses dialog(1) to show the progress instead:

((cd /; sudo tar cf - --one-file-system . ) | pv -n -s $( sudo du -sxm --apparent-size / | cut -f1 )m >/dev/null) 2>&1 | dialog --gauge 'Progress' 7 70

make sure to edit the new fstab!!! remove the / entry

vi /mnt/rastank/etc/fstab

create initramfs with zfs support

sudo update-initramfs -c -k $(uname -r)

verify zfs is in the initrd:

lsinitramfs /boot/initrd.img-4.14.30-v7+ | grep bin/z

add this to config.txt: initramfs initrd.img-4.14.30-v7+ followkernel

you must update this when a new kernel is installed

example cmdline.txt to boot from zfs pool, limit zfs ARC to 64M:

dwc_otg.lpm_enable=0 console=serial0,115200 root=ZFS=rastank/raspbian-stretch rootfstype=zfs elevator=noop rootwait plymouth.enable=0 zfs.zfs_arc_max=67108864

Other things to do:

  • you can use a usb drive and /dev/sda to boot from a thumb drive
  • After booting from zfs partition, destroy partition 2
  • copy files off of /boot, enlarge partition 1 and create a new vfat filesystem, copy files back
  • use remaining space from partition 2 as swap partition, or use another computer to move zfs partition next to partition 1
  • Create a swap zvol so you don't need to have a dedicated partition for swap (you can also make a swap dataset, do not use swap files on a normal dataset, pain will ensue.)

Problems:

  • You can enable a serial console or use hdmi and a usb keyboard.

ZFS files missing from initrd:

Gave up waiting for root file system device. Common problems:

  • Boot args (cat /proc/cmdline)

    • Check rootdelay= (did the system wait long enough?)
  • Missing modules (cat /proc/modules; ls /dev)

ALERT! ZFS=rastank/raspbian-stretch does not exist. Dropping to a shell!

(initramfs) [ 78.336935] random: crng init done

(initramfs) zpool

sh: 2: zpool: not found

Tell dkms to build and install the missing zfs module:

sudo dkms install zfs/0.7.6 -k 4.14.34-v7+

ZFS cache file out of date or drive is missing:

Command: /sbin/zpool import -c /etc/zfs/zpool.cache -N 'rastank'

Message: cannot import 'rastank': one or more devices is currently unavailable

Error: 1

Failed to import pool 'rastank'.

Manually import the pool and exit.

BusyBox v1.22.1 (Raspbian 1:1.22.0-19) built-in shell (ash)

Enter 'help' for a list of built-in commands.

/bin/sh: can't access tty; job control turned off

/ # /sbin/zpool import -c /etc/zfs/zpool.cache -N 'rastank'

cannot import 'rastank': one or more devices is currently unavailable

/ # **/sbin/zpool import -N 'rastank'**

/ # done.

/ # **exit**

done.

Begin: Mounting 'rastank/raspbian-stretch' on '/root//' ... done.

done.

Clone this wiki locally