Dual-Boot NetBSD And Archlinux

2024.07.18 20:32 NetBSD Linux UEFI encryption installation

This is the method I use to dual-boot Archlinux and NetBSD in my laptop. I’ve written this with a virtual machine, to reproduce and test every step. But, at least in theory, it should work in any UEFI AMD64 machine.

Motivation

I’ve been forcing myself to use NetBSD.

When you are familiar with a tool it’s very hard to get the work done with a different one. Working with a new and unfamiliar tool makes you slower, feels less ergonomic and frustrating. This is why people keep stuck with proprietary bloatware like Adobe Photoshop and alike. Not because the lack of open/libre alternatives, but because these alternatives are different and unfamiliar.

So, to learn a tool, you have to force yourself to use it. In your spare time, if your profession is merciless with mistakes and delays. In your working time, if your profession allows it.

Working with a new tool is painful and frustrating. So certain discipline is required.

But once the initial pain goes away, stuff becomes easier with time. And the new tool becomes a little bit more familiar every day.

Then, you can judge this tools by its own merits. With experience you can tell where this tool is stronger, and where is weaker.

With NetBSD I can use Darktable to edit and process my photos. Basic stuff like reading my e-mail and browse the web. I can use my favorite password manager and synchronize the keyring with my phone.

But I can’t use my Wacom tablet in NetBSD. While, I can do some photo editing with it, NetBSD is not really meant to do artwork with it. Probably, NetBSD has been heavily tested with server workloads, but not so much with desktop ones. And hardware support is not comparable with Linux. So, If you want to draw stuff with your computer, I wouldn’t recommend NetBSD.

So, in order to keep using NetBSD and have the ability to fallback to a more familiar OS when needed, I’ve decided to dual-boot Archlinux and NetBSD.

This is a tutorial that I’ve written to myself to do this.

Arch Linux Installation

This blog post have been written with a virtual machine to keep track of the steps and test them. In this kind of environment, is better to use the serial port for terminal, rather than the virtual screen.

Once the arch installer image boots, you may use the serial port.

# systemctl start serial-getty@ttyS0.service

This tutorial assumes two drives in a machine, which is the setup of my laptop. It could be done with a single drive though.

Let’s see what disks we have available:

# fdisk -l
Disk /dev/vda: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/vdb: 20 GiB, 21474836480 bytes, 41943040 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/loop0: 795.73 MiB, 834379776 bytes, 1629648 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Here I have two drives with 32GiB and 20GiB. The bigger one will have the EFI partition and Archlinux. The smaller one will have NetBSD installed.

The EFI partition is usually the first partition of the first drive. It’s a FAT partition that contains binaries that loads your OS. It may contain configuration and data files too.

In this partition we are going to put the boot-loaders for Archlinux and NetBSD. Also, here we are going to install rEFInd, a tool that helps us to manage and boot different OSes.

In theory, we could use the boot menu from the machine firmware. Or GRUB, or something similar. So rEFInd is not mandatory, but it will give us a nicer graphical boot menu to select what OS we want to use.

To proceed with Archlinux installation we may need to connect to WiFi. Remember, these are the steps for my laptop. Your setup could be different.

# iwctl station wlan0 scan
# iwctl station wlan0 get-networks
# iwctl --passphrase passphrase station wlan0 connect SSID

Now we create the partitions for the first drive.

# fdisk /dev/vda

Use g to create new GPT partition. Use n to create a new partition. t and 1 to change a partition to EFI type. And w to save and exit.

New partitions should look like this.

# fdisk -l /dev/vda
Disk /dev/vda: 32 GiB, 34359738368 bytes, 67108864 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 88E94B49-29C5-4B0D-B2D4-E9EB94CC52C7

Device        Start      End  Sectors Size Type
/dev/vda1      2048  8390655  8388608   4G EFI System
/dev/vda2   8390656 16779263  8388608   4G Linux filesystem
/dev/vda3  16779264 67106815 50327552  24G Linux filesystem

In this case I’ve created an EFI partition of 4GiB. A Linux /boot partition of 4GiB. And 24GiB for / and other filesystems of Archlinux.

4GiB is a lot of space for an EFI partition. You shouldn’t need more than that.

Now we create FAT filesystems for the /boot and EFI partitions.

# mkfs.vfat -n EFI /dev/vda1
mkfs.fat 4.2 (2021-01-31)
# mkfs.vfat -n BOOT /dev/vda2
mkfs.fat 4.2 (2021-01-31)

# blkid /dev/vda{1,2}
/dev/vda1: LABEL_FATBOOT="EFI" LABEL="EFI" UUID="0982-62D9" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="5ca2bc8a-7624-4b82-8097-b887256ece2d"
/dev/vda2: LABEL_FATBOOT="BOOT" LABEL="BOOT" UUID="0A17-5347" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="25e6e4e2-b4eb-4741-b998-b9d9c0a8bc5c"

Then we encrypt the Linux partition. In this setup we will use LVM on top of LUKS. LUKS will encrypt the partition. And LVM will make logical volumes out of that partition.

# cryptsetup luksFormat /dev/vda3
# cryptsetup open /dev/vda3 cryptlvm

Usually, I use 4 random words a la diceware for the passphrase. That’s not a very strong passphrase. But I believe it’s good enough for off-line stuff, like a encrypted drive.

For internet stuff, where malicious actors do have access to credentials sometimes, I use a password manager. And 24 to 32 random characters as the passwords. With 2FA when possible.

But for the drive encryption, I don’t recommend a password too hard to remember. And, unlike some advice on The Internet, I do recommend writing the passphrase in a piece of paper, and store it with other important documents. If you are not a state-level target, that’s good enough.

If you need a strong password, 7 diceware words should be enough for everybody.

Now, we create the LVM group with the logical volumes.

# pvcreate /dev/mapper/cryptlvm
# vgcreate arch /dev/mapper/cryptlvm
# lvcreate -L 4G arch -n swap
# lvcreate -L 16G arch -n root
# lvcreate -l 100%FREE arch -n home

Then, we create the filesystems for it.

# mkfs.ext4 -L ROOT /dev/arch/root
# mkfs.ext4 -L HOME /dev/arch/home
# mkswap /dev/arch/swap 
# swapon /dev/arch/swap

Then, mount everything under new rootfs. As the Archlinux installation guide suggest, we are going to use /mnt as the new root filesystem.

# mount /dev/arch/root /mnt
# mkdir -p /mnt/home       
# mkdir -p /mnt/boot
# mount /dev/arch/home /mnt/home
# mount /dev/vda2 /mnt/boot
# mkdir -p /mnt/boot/efi
# mount /dev/vda1 /mnt/boot/efi

Here, we are mounting the EFI filesystem in /boot/efi, so it can be manipulated from Archlinux. This is not mandatory. But I usually do it, so I can edit, backup, delete stuff easily.

Now, we install the system.

# pacstrap -K /mnt base base-devel linux linux-firmware iwd dhcpcd vim htop tmux grub efibootmgr os-prober lvm2

A lot of these packages are not mandatory. This is what a new system for me usually looks like. You could replace grub for other bootloader, since we are using rEFInd and we are not gonna use a lot of GRUB features. At minimum base, linux and lvm2 are necessary for this setup.

Now, let’s create a new fstab for our system.

# genfstab -U /mnt >> /mnt/etc/fstab

The last configs for our new system will require to chroot into the new system. Archlinux have a convenient script called arch-chroot that do other stuff, like mounting special filesystem into the new root.

# arch-chroot /mnt
# ln -sf /usr/share/zoneinfo/Europe/Madrid /etc/localtime
# hwclock --systohc
# vim /etc/locale.gen # Uncomment your locale
# locale-gen 
# echo LANG=en_US.UTF-8 >> /etc/locale.conf
# echo arch > /etc/hostname
# vim /etc/mkinitcpio.conf # Add hooks for LUKS and lvm2

The last part is quite important. If you do it wrong, your new system won’t boot

This is how it should look like.

# grep ^HOOKS /etc/mkinitcpio.conf
HOOKS=(base udev autodetect microcode modconf kms keyboard keymap consolefont block encrypt lvm2 filesystems fsck)

Now, we generate the new ramfs, and change the root user password.

# mkinitcpio -P
# passwd

Then, we install and configure the bootloader. As I said, you may want to use a smaller, leaner bootloader. We are using rEFInd, so our bootloader doesn’t need to chain-load other bootloaders.

# grub-install --target=x86_64-efi --efi-directory=/boot/efi --bootloader-id=GRUB
# blkid /dev/vda3 # Get the UUID of the encrypted drive                               
/dev/vda3: UUID="83e10b9c-2420-4b23-b8a5-3e0a09749f52" TYPE="crypto_LUKS" PARTUUID="5342f00d-3827-4ca1-abb8-96f0660267c7"
# vim /etc/default/grub # Add the kernel parameter to the encrypted drive
# grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 cryptdevice=UUID=83e10b9c-2420-4b23-b8a5-3e0a09749f52:cryptlvm root=/dev/arch/root"

We have to add a kernel parameter to decrypt the root filesystem. So your kernel parameters should look like this.

# grep GRUB_CMDLINE_LINUX_DEFAULT /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="loglevel=3 cryptdevice=UUID=83e10b9c-2420-4b23-b8a5-3e0a09749f52:cryptlvm root=/dev/arch/root"

Of course, you replace my partition UUID with yours. You print that with blkid.

If it’s good, we proceed to make the GRUB configuration.

# grub-mkconfig -o /boot/grub/grub.cfg

At this point, you can reboot and use your new Archlinux system. No graphical user interface, of course. That’s outside the scope of this guide. I usually install Mate or i3wm.

Now, the fun part. We need NetBSD in this computer too.

Install NetBSD

I’ve already written a guide to install NetBSD with encrypted partitions. This will be almost the same, with a few differences for EFI configuration.

The first thing: we need to know the available drives.

# sysctl hw.disknames
# dkctl ld0
# dkctl ld1

Of course, it puts different names for the drives. ld0 and ld1 are the name of my virtual drives in this machine. Yours may be different. Keep in mind those differences in names.

We need to make sure we are formatting and writing in the right drive.

# dkctl ld0 listwedges
# dkctl ld1 listwedges

Once you know, what drive is the right one, we need to create new partitions for it. Assuming ld1 is the right one.

# gpt destroy ld1
# gpt create ld1
# gpt add -a 2m -l NetBSD -t ffs -s 8g ld1
# gpt add -a 2m -t cgd -l syscgd ld1

Here stuff may get complicated. Since we have created new partitions, the numbers of dk* will change. So you have to list the wedges again and make sure that you are going to format the right ones.

# dkctl ld0 listwedges
/dev/rld0: 3 wedges:
dk2: 5ca2bc8a-7624-4b82-8097-b887256ece2d, 8388608 blocks at 2048, type: msdos
dk3: 25e6e4e2-b4eb-4741-b998-b9d9c0a8bc5c, 8388608 blocks at 8390656, type: ext2
fs
dk4: 5342f00d-3827-4ca1-abb8-96f0660267c7, 50327552 blocks at 16779264, type: ex
t2fs
# dkctl ld1 listwedges
/dev/rld1: 2 wedges:
dk0: NetBSD, 33554432 blocks at 4096, type: ffs
dk1: syscgd, 8380416 blocks at 33558528, type: cgd

See? dk0 is now the NetBSD root filesystem. And dk1 will be the NetBSD encrypted drive with protected filesystems.

Now, let’s add an EFI entry for NetBSD.

# mount /dev/dk2 /mnt
# mkdir -p /mnt/EFI/NetBSD/
# cp -v /usr/mdec/*.efi /mnt/EFI/NetBSD
/usr/mdec/bootia32.efi -> /mnt/EFI/boot/bootia32.efi
/usr/mdec/bootx64.efi -> /mnt/EFI/boot/bootx64.efi

And we will create an EFI config file for NetBSD

vi /mnt/boot.cfg

menu=Boot normally:root NAME=NetBSD;boot
menu=Boot single user:root NAME=NetBSD;boo -s
menu=Disable ACPI:root NAME=NetBSD;boot -2
menu=Disable ACPI and SMP:root NAME=NetBSD;boot -12
menu=Drop to boot prompt:prompt
default=1
timeout=5
clear=1

The important part is root NAME=NetBSD. We are telling the NetBSD bootloader that the root filesystem is in a partition labeled NetBSD.

Now we format the partition and mount it under /targetroot.

# newfs -O 2 dk0
# mount /dev/dk0 /targetroot

Then, proceed with the new encrypted CGD device.

# mkdir -p /targetroot/etc/cgd/
# cgdconfig -g -V disklabel -o /targetroot/etc/cgd/syscgd aes-xts 512
# cgdconfig -V re-enter cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd
# disklabel -Ii cgd0
# echo 'cgd0 NAME=syscgd /etc/cgd/syscgd' > /targetroot/etc/cgd/cgd.conf

These steps are better explained in my last blog entry, and the documentation. After the disklabel -Ii cgd0 part you should have the partitions for the protected CGD device. I usually use cgd0a for /var, cgd0b for swap, cgd0e for /usr, and cgd0f for /home.

Let’s test our new CGD device. Unconfigure (close) the CGD device, then configure it again. Then, print the disklabel.

# cgdconfig -u cgd0
# cgdconfig cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd
# disklabel cgd0

If everything is OK, we create and mount our new filesystems under /targetroot.

# newfs -O 2 cgd0a
# newfs -O 2 cgd0e
# newfs -O 2 cgd0f
# mkdir /targetroot/var /targetroot/usr /targetroot/home
# mount /dev/cgd0a /targetroot/var
# mount /dev/cgd0e /targetroot/usr
# mount /dev/cgd0f /targetroot/home

With the partitions mounted, we can extract the binary sets.

# cd /amd64/binary/sets
# tar xvzpf base.tar.xz -C /targetroot
# tar xvzpf comp.tar.xz -C /targetroot
# tar xvzpf etc.tar.xz -C /targetroot
# tar xvzpf games.tar.xz -C /targetroot
# tar xvzpf gpufw.tar.xz -C /targetroot
# tar xvzpf kern-GENERIC.tar.xz -C /targetroot
# tar xvzpf man.tar.xz -C /targetroot
# tar xvzpf misc.tar.xz -C /targetroot
# tar xvzpf modules.tar.xz -C /targetroot
# tar xvzpf text.tar.xz -C /targetroot
# tar xvzpf xbase.tar.xz -C /targetroot
# tar xvzpf xcomp.tar.xz -C /targetroot
# tar xvzpf xetc.tar.xz -C /targetroot
# tar xvzpf xfont.tar.xz -C /targetroot
# tar xvzpf xserver.tar.xz -C /targetroot
# cd /

Then, chroot to new system and make the devices in /dev.

# chroot /targetroot
# cd dev
# ./MAKEDEV all
# exit

Now, we edit fstab.

# vi /targetroot/etc/fstab

# NetBSD /etc/fstab
# See /usr/share/examples/fstab/ for more examples.
NAME=NetBSD     /       ffs     rw              1 1
kernfs          /kern   kernfs  rw
ptyfs           /dev/pts        ptyfs   rw
procfs          /proc   procfs  rw
/dev/cd0a       /cdrom  cd9660  ro,noauto
tmpfs           /var/shm        tmpfs   rw,-m1777,-sram%25

# Encrypted file-systems
/dev/cgd0a      /var    ffs     rw      1 2
/dev/cgd0b      none    swap    sw
/dev/cgd0e      /usr    ffs     rw      1 2
/dev/cgd0f      /home   ffs     rw      1 2

Then we edit rc.conf.

# vi /targetroot/etc/rc.conf

rc_configured=YES
critical_filesystems_local="OPTIONAL:/var OPTIONAL:/usr"

# Add local overrides below.
dhcpcd=YES
dhcpcd_flags="-qM wm0"
hostname=ceres.local
sshd=YES
wscons=YES
cgd=YES

Then we add the kernel to EFI partition. You see, this part may not be needed.

Maybe because of UEFI firmware issues, the second drive was not available always in bootloader runtime. So, I couldn’t read the kernel from the second drive. I added the kernel to the EFI drive and, if you recall it, we told the bootloader to find root filesystem in root NAME=NetBSD.

cp -v /targetroot/netbsd /mnt

Now, let’s umount the new system.

# umount /targetroot/home
# umount /targetroot/var
# umount /targetroot/usr
# umount /targetroot/
# umount /mnt/

Then shutdown or reboot:

# shutdown -p now

Install rEFInd

At this point, you have Archlinux and NetBSD installed in your system. you could add a new menuentry in GRUB to access NetBSD. Or you could use the boot menu from your machine firmware.

I will use rEFInd to have a nice menu where I can select the OS.

Let’s boot in Archlinux and install rEFInd.

# pacman -S refind
# refind-install

Then, reboot.

Conclusion

I’m still working on teaching myself NetBSD. Linux ecosystem have become increasingly more bloated and bigger over time. And I’m sure that there are some scenarios where a simpler compact system would suit better.

But I’m too familiar with Linux, so I have to go outside my comfort zone. NetBSD is an amazing OS. Simple, consistent, documented and straightforward. But I have to use it more, so it becomes familiar to me.

Dual-boot is a good way to force yourself to use a different OS, with the fallback in case you need your familiar one. That’s why a lot of people dual-boot Windows and some Linux distro.