Setting up ZFS with Arch root install
Configuration overview
The last setup of this machine was an UNRAID install with virtual machines using PCI passthrough. For this setup I am going to run Arch Linux with a root install on ZFS. The root install will allow snapshots of the entire operating system.
Array hardware
- Boot Drive - 32 GB USB flash drive
- Array Storage - 3x 3TB HDD
- Cache - 2x 128GB SSD
Final configuration
The completed setup will have the following zpool configuration and mount points
$ zpool status
pool: vault
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
vault ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-ST3000DM001-1CH144_Z1F5W372 ONLINE 0 0 0
ata-ST3000DM001-1CH144_Z1F5YJ5C ONLINE 0 0 0
ata-ST3000DM001-1C6144_Z1F5KYV4 ONLINE 0 0 0
cache
ata-Samsung_SSD_850_EVO_120GB_S21TNSAG205110A ONLINE 0 0 0
ata-Samsung_SSD_850_EVO_120GB_S21WNX0H404232B ONLINE 0 0 0
errors: No known data errors
$ zfs mount
vault/ROOT/default /
vault/home /home
vault /vault
Create EFI partition on boot drive
Boot into archlive in UEFI mode and create a 512MB EFI partition on the USB drive
# enter into fdisk for the required device
$ fdisk /dev/sdX
$ g # create a new GPT partition table
$ n # create a new partition
$ enter # select partition 1 as default
$ enter # select default start location
$ +512M # end location of 512M after start location
$ t # change the partition type to EFI
$ 0 # in fdisk the EFI system partition is 0
$ w # write changes to disk
Create and configure the pool
You do not need to partition the disks that will be used in the pool as zfs will
partition the drives automatically when the pool is created.
Use disk id to specify the disks in the pool. You can find disk ids with the
command ls /dev/disk/by-id. This install is going to use a 3 way mirror with
the SSDs acting as the cache.
Load the kernel module
$ modprobe zfs
Note the -o ashift=12 will use 4096 byte sectors which is what you want
for modern disks.
$ zpool create -f -o ashift=12 vault mirror \
ata-ST3000DM001-1CH144_Z1F5W372 \
ata-ST3000DM001-1CH144_Z1F5YJ5C \
ata-ST3000DM001-1C6144_Z1F5KYV4 \
cache \
ata-Samsung_SSD_850_EVO_120GB_S21TNSAG205110A \
ata-Samsung_SSD_850_EVO_120GB_S21WNX0H404232B
Check the status of the pool to make sure it was created correctly.
$ zpool status
pool: vault
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
vault ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
ata-ST3000DM001-1CH144_Z1F5W372 ONLINE 0 0 0
ata-ST3000DM001-1CH144_Z1F5YJ5C ONLINE 0 0 0
ata-ST3000DM001-1C6144_Z1F5KYV4 ONLINE 0 0 0
cache
ata-Samsung_SSD_850_EVO_120GB_S21TNSAG205110A ONLINE 0 0 0
ata-Samsung_SSD_850_EVO_120GB_S21WNX0H404232B ONLINE 0 0 0
errors: No known data errors
Turn on compression and optimize writes to disk with relatime
#turn on compression
$ zfs set compression=on vault
# optimize writes to disk
$ zfs set relatime=on vault
Create datasets for / and home
$ zfs create -o mountpoint=/ vault/ROOT/default
$ zfs create -o mountpoint=/home vault/home
Specify the dataset that will be used to boot from
$ zpool set bootfs=vault/ROOT/default vault
Unmount zfs volumes and export before installing arch
$ zfs umount -a
$ zpool export vault
Install the base Arch system
Create directories for mountpoints
$ mkdir /mnt/{boot,home}
Mount zfs and boot volumes
$ mount /dev/sdX1 /mnt/boot # boot disk
$ zpool import -d /dev/disk/by-id -R /mnt vault
$ pacstrap -i /mnt base base-devel
Copy the zpool.cache file to the arch install
$ cp /etc/zfs/zpool.cache /mnt/etc/zfs/zpool.cache
Generate the fstab and make sure /boot is mounted
$ genfstab -U -p /mnt >> /mnt/etc/fstab
Add hooks to mkinitcpio.conf and regenerate it
$ vim /mnt/etc/mkinitcpio.conf
# Add zfs after keyboard but before filesystems
HOOKS="base udev autodetect modconf block keyboard zfs filesystems"
# regenerate mkinitcpio
$ mkinitcpio -p linux
Chroot into the Arch install and configure the system
$ arch-chroot /mnt /bin/bash
Add the archzfs repo to /etc/pacman.conf
$ vim /etc/pacman.conf
# add the following in the repository section
[archzfs]
Server = http://archzfs.com/$repo/x86_64
Sign the repository key with the key from AUR
$ pacman-key -r 5E1ABF240EE7A126
$ pacman-key --lsign-key 5E1ABF240EE7A126
Update the system and install zfs
$ pacman -Syyu
$ pacman -S zfs-linux
Enable zfs services
$ systemctl enable zfs.target
$ systemctl enable zfs-import-cache.service
Install the EFI bootloader
bootctl --path=/boot install
Create an entry for Arch in the bootloader
$ vim /boot/loader/entries/arch.conf
# add the following
title Arch Linux
linux /vmlinuz-linux
initrd /initramfs-linux.img
options zfs=vault/ROOT/default rw
Exit chroot unmount drives and restart
# exit chroot before unmounting
$ umount /mnt/boot
$ zfs umount -a
$ zpool export vault
$ reboot
Apparently when the computer is booting the host id is not available to the system. To fix this, create a hostid file and regenerate mkinitcpio. After rebooting:
$ hostid > /etc/hostid
$ mkinitcpio -p linux