-
Notifications
You must be signed in to change notification settings - Fork 55
HOWTO install Raspbian to a Native ZFS Root Filesystem, or, How I Learned to Love Data Integrity
Get the OS onto a card with room to spare for zfs, or you can use/boot a USB drive with only /boot on the SD Card
- Write OS to sd card, edit config.txt. Remove init=/usr/lib/raspi-config/init_resize.sh
- create “ssh” file in boot partition to enable ssh
Boot the rpi
sudo vi /etc/apt/sources.list
sudo apt-get update ; sudo apt-get -y dist-upgrade
Remove unused kernel to make room on /boot and / (We will be very tight on space with a fresh, unexpanded install)
sudo rm /boot/kernel.img
sudo rm -rf /lib/modules/4.14.30+
sudo apt-get -y install build-essential fakeroot devscripts autogen libelf-dev debian-keyring busybox initramfs-tools
sudo apt-get clean
sudo apt-get -y build-dep zfs-dkms spl-dkms
mkdir ~/src ; cd ~/src
dget -x http://http.debian.net/debian/pool/main/s/spl-linux/spl-linux_*.dsc
dget -x http://http.debian.net/debian/pool/contrib/z/zfs-linux/zfs-linux_*.dsc
curl "https://bugs.debian.org/cgi-bin/bugreport.cgi?att=1;bug=826994;filename=sysvinit_patch_5_826994.diff;msg=112" > init.patch
curl https://github.com/zfsonlinux/zfs/commit/7403d0743e2b75b7f5412a14007ba159efb67a7d.patch > ob1.patch
cd zfs-linux-0.7.6 ; cat ../init.patch ../ob1.patch | patch -p1 ; cd ..
cd spl-linux-0.7.6 ; debuild -i -us -uc -b --no-lintian ; cd ..
cd zfs-linux-0.7.6 ; dpkg-source --commit ; debuild -i -us -uc -b --no-lintian ; cd ..
sudo apt-get install raspberrypi-kernel-headers
sudo apt-get clean
sudo dpkg -i spl_*_armhf.deb spl-dkms_*_all.deb
sudo swap off -a ; sudo rm /var/swap
sudo dpkg -i libnvpair1linux_*_armhf.deb libuutil1linux_*_armhf.deb libzfs2linux_*_armhf.deb libzpool2linux_*_armhf.deb zfs-dkms_*_all.deb zfsutils-linux_*_armhf.deb zfs-zed_*_armhf.deb zfs-initramfs_*.deb
reboot
lsmod | grep zfs
modprobe zfs
create a zfs pool on the SD Card (USB drives would be sda):
sudo fdisk /dev/mmcblk0
p (print part table, look when p2 ends (3629055 on mine)
Device Boot Start End Sectors Size Id Type
/dev/mmcblk0p1 8192 93802 85611 41.8M c W95 FAT32 (LBA)
/dev/mmcblk0p2 98304 3629055 3530752 1.7G 83 Linux
n (new)
p (primary)
3
First sector is end of p2+1 = 3629056
end is default or +xG to make a x GB partition
w (to write partition table, likely need to reboot to reread the partition table)
Now create a pool, set the temp mount root to /mnt/rastank, and create a root dataset for easy snapshotting and rollbacks. It’s always a good idea to never put files in the pool without a dataset.
sudo zpool create -o ashift=12 -O canmount=off -O compression=lz4 -O normalization=formD -O mountpoint=/mnt/rastank -O xattr=sa -O relatime=on -O recordsize=1M -R /mnt/rastank rastank /dev/mmcblk0p3
sudo zfs create -o mountpoint=/ rastank/raspbian-stretch
Now you can copy your system to the new ZFS partition. Use tar to preserve hard links and any special files. Install pv and di as helpful status utilities. If you don't want the progress to show, pipe the first tar command to the other.
sudo apt-get install pv di dialog
(cd /; sudo tar cf - --one-file-system . ) | pv -p -bs $( sudo du -sxm --apparent-size / | cut -f1 )m | (sudo tar -xp /mnt/rastank)
If you want it even more pretty, use this command and it uses dialog(1) to show the progress instead:
((cd /; sudo tar cf - --one-file-system . ) | pv -n -s $( sudo du -sxm --apparent-size / | cut -f1 )m >/dev/null) 2>&1 | dialog --gauge 'Progress' 7 70
vi /mnt/rastank/etc/fstab
sudo update-initramfs -c -k $(uname -r)
lsinitramfs /boot/initrd.img-4.14.30-v7+ | grep bin/z
dwc_otg.lpm_enable=0 console=serial0,115200 root=ZFS=rastank/raspbian-stretch rootfstype=zfs elevator=noop rootwait plymouth.enable=0 zfs.zfs_arc_max=67108864
- you can use a usb drive and /dev/sda to boot from a thumb drive
- After booting from zfs partition, destroy partition 2
- copy files off of /boot, enlarge partition 1 and create a new vfat filesystem, copy files back
- use remaining space from partition 2 as swap partition, or use another computer to move zfs partition next to partition 1
- Create a swap zvol so you don't need to have a dedicated partition for swap (you can also make a swap dataset, do not use swap files on a normal dataset, pain will ensue.)
- You can enable a serial console or use hdmi and a usb keyboard.
Gave up waiting for root file system device. Common problems:
-
Boot args (cat /proc/cmdline)
- Check rootdelay= (did the system wait long enough?)
-
Missing modules (cat /proc/modules; ls /dev)
ALERT! ZFS=rastank/raspbian-stretch does not exist. Dropping to a shell!
(initramfs) [ 78.336935] random: crng init done
(initramfs) zpool
sh: 2: zpool: not found
sudo dkms install zfs/0.7.6 -k 4.14.34-v7+
Command: /sbin/zpool import -c /etc/zfs/zpool.cache -N 'rastank'
Message: cannot import 'rastank': one or more devices is currently unavailable
Error: 1
Failed to import pool 'rastank'.
Manually import the pool and exit.
BusyBox v1.22.1 (Raspbian 1:1.22.0-19) built-in shell (ash)
Enter 'help' for a list of built-in commands.
/bin/sh: can't access tty; job control turned off
/ # /sbin/zpool import -c /etc/zfs/zpool.cache -N 'rastank'
cannot import 'rastank': one or more devices is currently unavailable
/ # **/sbin/zpool import -N 'rastank'**
/ # done.
/ # **exit**
done.
Begin: Mounting 'rastank/raspbian-stretch' on '/root//' ... done.
done.