NetBSD - UEFI installation with Full Disk Encryption
NetBSD UEFI encryption installationI’ve been trying to teach myself NetBSD. It has been a painful experience, full of bugs and kernel panics. I’ve post some of them in the Fediverse. Like here and here.
One of the weakest points that I’ve seen in NetBSD is the installer. If you need a simple installation it just works ™. But as soon as you need some complex setup, like RAID mixed with encrypted partitions, or something similar, the installer is subpar. You will face some segfault from the installer, a kernel panic or another surprise.
I’ve forced myself to use NetBSD as a daily driver in my laptop. Since it’s a laptop it needs full disk encryption. A mobile computer can be stolen at anytime, specially if you live in the Lawlessness. So I had to make a “special way” to install NetBSD with encryption.
Now, even today I have no idea how to have actual full disk encryption with NetBSD.
CGD devices, the virtual block devices that implements encryption, require metadata in /etc/cgd.
I’ve seen a wonderful tutorial for full-disk encryption for MBR-based systems, but not for UEFI.
And my laptop doesn’t really like to boot in old MBR mode.
Following the documentation on CGD drives and the documentation on UEFI installations, I have a semi-full disk encryption.
With plain-text root file-system, and encrypted /home, /usr, /var and swap.
So, without further complaints, this is the way I have some disk encryption in UEFI systems with NetBSD.
Boot the installer image
I wrote this post with a virtual machine to reproduce all the steps.
In such environment it may be useful to use the serial port as a terminal instead the virtual monitor.
So, as soon as the boot options are presented, I press option 3.
Then, I instruct the boot-loader to use com0 as the terminal.
consdev com0
boot
Use a shell
Once the installer system is up, I avoid the installer and use a shell.
Select Utility menu, then Run /bin/sh.
Disks
Now that I’m in a shell I can look what disks I have.
# sysctl hw.disknames
hw.disknames = cd0 wd0
In this example wd0 is the drive where the system should be installed.
Since I want to boot with UEFI, it needs GPT partitions.
# gpt destroy wd0
# gpt create wd0
# gpt add -a 2m -s 128m -t efi -l EFI wd0
# gpt add -a 2m -l NetBSD -t ffs -s 8g wd0
# gpt add -a 2m -t cgd -l syscgd wd0
First, I destroy some pre-existing GPT partitions. This will destroy the drive data, so be careful.
These commands create three GPT partitions. One EFI partition with 128MB, one for the rootfs with 8GB and other for CGD with the rest of the disk.
These are the sizes I use. Root file-system will require less than 8GB anyways, but I avoid the risk.
Now this disk should have three wedges.
# dkctl wd0 listwedges
/dev/rwd0: 3 wedges:
dk0: EFI, 262144 blocks at 4096, type: msdos
dk1: NetBSD, 16777216 blocks at 266240, type: ffs
dk2: syscgd, 24895488 blocks at 17043456, type: cgd
In this example, /dev/dk0 is the EFI partition, /dev/dk1 is the root partition, and /dev/dk2 is the encrypted partition.
You may also run:
# gpt show wd0
start size index contents
0 1 PMBR
1 1 Pri GPT header
2 32 Pri GPT table
34 4062 Unused
4096 262144 1 GPT part - EFI System
266240 16777216 2 GPT part - NetBSD FFSv1/FFSv2
17043456 24895488 3 GPT part - NetBSD Cryptographic Disk
41938944 4063 Unused
41943007 32 Sec GPT table
41943039 1 Sec GPT header
Now, I create a file-system for boot EFI partition. Its type is FAT.
# newfs_msdos /dev/rdk0
Then, I mount it and copy the .efi boot entries.
# mount /dev/dk0 /mnt
# mkdir -p /mnt/EFI/boot
# cp -v /usr/mdec/*.efi /mnt/EFI/boot
/usr/mdec/bootia32.efi -> /mnt/EFI/boot/bootia32.efi
/usr/mdec/bootx64.efi -> /mnt/EFI/boot/bootx64.efi
# umount /mnt
Then, I format NetBSD root partition.
The -O 2 flag means that it will use FFv2 file-system.
newfs -O 2 dk1
Now I can mount the future root file-system.
The installer uses /targetroot directory to install.
So, I’ll do the same.
# mount /dev/dk1 /targetroot
Next, I create CGD config file.
And I place it in the future /etc/cgd/ directory.
# mkdir -p /targetroot/etc/cgd/
# cgdconfig -g -V disklabel -o /targetroot/etc/cgd/syscgd aes-xts 512
Now it’s time to create the CGD drive.
# cgdconfig -V re-enter cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd
Here, the -V re-enter flag is needed to validate the encrypted drive without the disklabel, which doesn’t exist yet.
Once the CGD drive is configured it is possible to create the disklabel for it.
In this example, I use cgd0a for /var, cgd0b for swap, cgd0e for /usr, and finally cgd0f for /home.
Partitions c and d have a special meaning in NetBSD, so I’m not using them.
Because this is a example virtual machine I use very humble values of 3 and 4GB.
# disklabel -Ii cgd0
Enter '?' for help
partition>a
Filesystem type [4.2BSD]:
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: 3G
Partition size ('$' for all remaining) [12156c, 24895488s, 12156M]: ^C
# disklabel -Ii cgd0
Enter '?' for help
partition>a
Filesystem type [4.2BSD]:
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]:
Partition size ('$' for all remaining) [12156c, 24895488s, 12156M]: 3G
a: 6291456 0 4.2BSD 0 0 0 # (Cyl. 0 - 3071)
partition>b
Filesystem type [unused]: swap
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: a
Partition size ('$' for all remaining) [0c, 0s, 0M]: 2G
b: 4194304 6291456 swap # (Cyl. 3072 - 5119)
partition>e
Filesystem type [unused]: 4.2BSD
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: b
Partition size ('$' for all remaining) [0c, 0s, 0M]: 3G
e: 6291456 10485760 4.2BSD 0 0 0 # (Cyl. 5120 - 8191)
partition>f
Filesystem type [unused]: 4.2BSD
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: e
Partition size ('$' for all remaining) [0c, 0s, 0M]: $
f: 8118272 16777216 4.2BSD 0 0 0 # (Cyl. 8192 - 12155)
partition>W
Label disk [n]?y
Label written
partition>Q
Let’s add CGD config file to the future root file-system.
# echo 'cgd0 NAME=syscgd /etc/cgd/syscgd' > /targetroot/etc/cgd/cgd.conf
Now it’s time to check if the CGD drive is working as expected. I unconfigure and reconfigure it, then print the disklabel.
# cgdconfig -u cgd0
# cgdconfig cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd
# disklabel cgd0
The new partitions need to be formatted and mounted in their places.
# newfs -O 2 cgd0a
# newfs -O 2 cgd0e
# newfs -O 2 cgd0f
# mkdir /targetroot/var /targetroot/usr /targetroot/home
# mount /dev/cgd0a /targetroot/var
# mount /dev/cgd0e /targetroot/usr
# mount /dev/cgd0f /targetroot/home
At this time, the system should look like this:
# mount
root_device on / type cd9660 (read-only, local)
tmpfs on /dev type tmpfs (union, local)
tmpfs on /tmp type tmpfs (local)
tmpfs on /var type tmpfs (local)
tmpfs on /etc type tmpfs (union, local)
/dev/dk1 on /targetroot type ffs (local)
/dev/cgd0a on /targetroot/var type ffs (local)
/dev/cgd0e on /targetroot/usr type ffs (local)
/dev/cgd0f on /targetroot/home type ffs (local)
Installation of the sets
The new system is composed of sets. I usually install these ones, but your requirements may be different.
The tar flag p is very important, since these files need to preserve their owners and mode.
# cd /amd64/binary/sets
# tar xvzpf base.tar.xz -C /targetroot
# tar xvzpf comp.tar.xz -C /targetroot
# tar xvzpf etc.tar.xz -C /targetroot
# tar xvzpf games.tar.xz -C /targetroot
# tar xvzpf gpufw.tar.xz -C /targetroot
# tar xvzpf kern-GENERIC.tar.xz -C /targetroot
# tar xvzpf man.tar.xz -C /targetroot
# tar xvzpf misc.tar.xz -C /targetroot
# tar xvzpf modules.tar.xz -C /targetroot
# tar xvzpf text.tar.xz -C /targetroot
# tar xvzpf xbase.tar.xz -C /targetroot
# tar xvzpf xcomp.tar.xz -C /targetroot
# tar xvzpf xetc.tar.xz -C /targetroot
# tar xvzpf xfont.tar.xz -C /targetroot
# tar xvzpf xserver.tar.xz -C /targetroot
# cd /
Then I chroot to the new system, and make the devices.
# chroot /targetroot
# cd dev
# ./MAKEDEV all
exit
Now I edit fstab to mount the CGD partitions.
# vi /targetroot/etc/fstab
# NetBSD /etc/fstab
# See /usr/share/examples/fstab/ for more examples.
NAME=NetBSD / ffs rw 1 1
kernfs /kern kernfs rw
ptyfs /dev/pts ptyfs rw
procfs /proc procfs rw
/dev/cd0a /cdrom cd9660 ro,noauto
tmpfs /var/shm tmpfs rw,-m1777,-sram%25
# Encrypted file-systems
/dev/cgd0a /var ffs rw 1 2
/dev/cgd0b none swap sw
/dev/cgd0e /usr ffs rw 1 2
/dev/cgd0f /home ffs rw 1 2
The file rc.confg also needs to be edited
# vi /targetroot/etc/rc.conf
rc_configured=YES
# Add local overrides below.
critical_filesystems_local="OPTIONAL:/var OPTIONAL:/usr"
hostname=marte.local
dhcpcd=YES
dhcpcd_flags="-qM wm0"
sshd=YES
wscons=YES
cgd=YES
rc_configured=YES is important, otherwise the system will always boot in single-user mode.
These are the variables I use for a new system.
For example, my network device is wm0.
And this example hostname is marte.local.
Your network card and requirements may be different.
Also critical_filesystems_local variable is quite important. Since our /usr filesystem is not mounted at the beginning, some tools may not be available. We need to wait for it to be mounted before executing binaries from /usr.
Unmount and reboot
Umount the new system:
# umount /targetroot/home
# umount /targetroot/var
# umount /targetroot/usr
# umount /targetroot/
Then shutdown or reboot:
# shutdown -p now
Fun fact. Sometimes at this point I experience a kernel panic. I have no idea why. It’s very unpredictable. Sometimes it happens, sometimes it doesn’t.
The new system
In the new system you may need to change root password:
# passwd
Install pkgin.
PATH="/usr/pkg/sbin:/usr/pkg/bin:$PATH"
PKG_PATH="http://cdn.netbsd.org/pub/pkgsrc/packages/NetBSD/amd64/10.0/All/"
# export PATH
# export PKG_PATH
# pkg_add pkgin
And add a new user.
# useradd -m -G wheel -k /etc/skel vsis
And, of course, RTFM.
# man afterboot
Conclusion
This is the method I use to install a semi-full disk encrypted NetBSD system.
I may add RAID devices, LVM, multiple disks, etc.
Then mount everything under /targetroot and extract the sets.
The restriction is in the root file-system. It needs to be in plain-text and in a regular partition. It seems to me that rootfs in CGD or LVM is not well supported.
I may be biased by the Arch way to install the system, but I find this method better than the installer.