Instructions on installing and settings up Debian NAS
These are instructions that can teach you to install a Debian system on a NAS.
I’m using Windows Hyper-V for testing this out. If you’re not using Windows,
it should be relatively easy to follow along on some other OS.
I’ll be installing onto a single disk, using: “password” as the password
everywhere and “testuser1” as the user, and I’ll be providing instructions for
changing passwords and keys and add more disks and setup redundancy from there.
Installation
This section describes what to do when you’re booted from the install media,
partitioning, filesystem creation, OS package fetching and installation and so
on. You can see how to login in firstboot section.
Using Hyper-V (testing recommended before attempting on real hardware/data)
- create an expanding 127G vhdx disk - nas disk1.vhdx
- generation 2 vm
- 1GB ram is enough, but disable “dynamic memory” - it can be too aggresive and
cause your VM to run out of ram mid way through partitioning or creating a
filesystem. - Bridge Switch networking (TODO, describe the setup).
- Use an existing virtual hard disk nas disk 1
- Find debian nas … Actions > Settings
- Firmware > Boot Order > DVD Drive > Move Up
- Security > Secure Boot > Template > Microsoft UEFI Certificate
Authority - SCSI controller - add DVD drive, pick debian netinst downloaded image
once you start the vm. you’ll see a boot menu Graphical install Install (choose
this one)
…
There’s a quirk with Hyper-V and keyboards in initramfs, see troubleshooting
below when you get to first actual boot.
Easy way using debian installer and local network
The install media we’re starting with is the “netinst” iso that debian provides.
You can go through most of this without resorting to actually typing commands,
it’s really nice. The partitioning/formatting tool built into the installer is
pretty good. It’ll install some stuff you won’t be using and won’t install all
the stuff you most likely need, but is pretty good overall.
It’ll start by asking you about lanugages, autodetecting your network, and
download some stuff, ask you the hostname, domain name, root password countries
and timezones
At some point you can choose partitions:
- Choose your empty disk and format it as GPT
- Start by making a 512M ESP partition which will store grub and its configs.
- Continue by making a 2GB btrfs /boot partition (unencrypted) for your kernel.
- Then make a “physical volume for encryption”, this will be your third
partition, that will use dm-crypt with aes 256 xts-plain64 format. - Now, choose “Configure encrypted volumes” which will commit your current
partitions and create an encrypted volume for configuring. It’ll ask ask you
for a passphrase, which you can configure here and change later. - Once this is commited, you’ll be back in the partitioner, you should see
“encrypted volume” show up above (or below?) your physical disk. Since this
is just an empty volume and not a usable root filesystem, set this encrypted
volume partition to btrfs and set mountpoint to “/”. - Choose to option to finish partitioning ignore the warning about not having
swap space. It’s just not all that useful. - Choose to write changes to disk.
At the end we’re aiming for a partitioning scheme that looks like this one:
root@debiannas:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 127G 0 disk
├─sda1 8:1 0 487M 0 part /boot/efi
├─sda2 8:2 0 1.9G 0 part /boot
└─sda3 8:3 0 124.7G 0 part
└─sda3_crypt 254:0 0 124.7G 0 crypt /
sr0 11:0 1 336M 0 rom
Soon after, the installer will start to download and install packages from the
network onto the actual system, and if you press Alt+F5 you can see
debootstrap unpacking all kinds of things. You can go back to first “virtual
console” where the installer is running by pressing Alt+F1, eventually it’ll
ask you if you want to “Scan another cd/dvd” - ofcourse not.
You’ll also be asked to pick a mirror for APT - the debian package manager your
server will be using, and whether to participate in the package user survey,
which you can skip.
You’ll be asked if you want a Debian Desktop environment, deselect it with
<space>, also deselect the “Print server” and select only “SSH server” and
standard system utilities (this is probably more than you strictly need, but
there might be some nice to haves in there for a server as well, such as vi
and nano text editors).
Eventually it’ll ask you to reboot. Which you can try doing, and you’ll be able
to log in, assuming you have a working keyboard and screen to see what you’re
doing (for Hyper-V testing, see Hyper-V keyboard below).
Congratulations !
Hard (no-nonsense way), using debootstrap
Extra hard with bonus points - headless over ssh
First boot
Once the system has booted “under its own kernel”, and you can unlock the root
partition via the local terminal, and finish booting.
Unfortunately that’s not the end of it, indeed there’s plenty of updating and
setup that still needs to be done, and more than on reboot may eventually be
necessary.
The kernel will start booting and you’ll be asked to enter a pasword password
to unlock the sda3_crypt volume, which is your root filesystem. After that, you
can login with root, password password.
First thing to do should be to enable SSH access so you can configure the rest
of the system from your comfy sofa, and not hunched over a keyboard in your
garage or poorly lit basement, or next to a noisy rack or whereever you keep
your server.
You’ll also need some things from your client for this.
Static IPs for the server and /etc/hosts entries for the client.
To make our lives easier, we’ll be using a static ip for our server on the local
network. Assuming our server is called debiannas and assuming it has an ip of
10.2.3.4 we can add that to our Hosts File.
10.2.3.4 debiannas
On Windows, you can find this in C:\Windows\System32\Drivers\etc\hosts.
On Linux, you can find this in /etc/hosts.
On MacOS you can open /etc/hosts and read the location /usr/private/..somwhere.
You can verify this worked by issuing ping debiannas and you should get
replies from your server (if you’re not on Windows, Ctrl+C will make it stop)
Setting a static IP address on the server.
Whichever DHCP server we’re running on the network, will probably try to
conserve the IP address space by giving the same host (as identified by the mac
address) the same IP address. We should ensure we always have the same address
by just configuring the network interface to use a static ip.
On Debian, we do this by changing eth0 in /etc/network/interfaces, where we
used to have:
allow-hotplug eth0
iface eth0 inet auto
we should have:
allow-hotplug eth0
iface eth0 inet static
address 10.2.3.4/24
gateway 10.2.3.1
We can also set our DNS parameters in /etc/resolv.conf
Enabling SSH Access
Since you installed “SSH server”, it should already be started. You can check by
running systemctl status once logged in locally.
Since password auth is not recommended in general, we’ll use basic public key
auth. We’ll be configuring this public key for the root account (ok for a
file-server you won’t be using for your day-to-day linux computing as a regular
user - since basically you’ll need root anyway every time you log in.)
Note: dropbear doesn’t support newer ed25519 keys, but works with ecdsa. If
you don’t already have an ecdsa ssh key, you should generate one now to be able
to unlock your disks remotely later.
Generating ECDSA keypair
You can run ssh-keygen.exe -t ecdsa -b 521 which will generate a pair of keys
you can use to both admin this server and then remote unlock it as needed. You
don’t need to set a key password for testing.
From a crypto perspective, as of early 2021 the 521 bit ECDSA keys are not a bad
choice in general. Naturally, in some kind of perfect world ssh certs would be
supported everywhere and we’d have redundant and secure and easy to keep safe
hardware tokens for all our secrets. Keeping your secrets secret is out of scope
of this guide :).
Transfering the public key
Turns out we get nc with default Debian install, and on Windows and many other
systems we have telnet (or at least telnet client is installable as a
component/feature ). We can just transfer the public key in plain text.
On the NAS host, run the command ip addr to show your ip address, then run nc -t -l -p 12345 | tee .ssh/authorized_keys to receive the key into authorized
keys file.
On the Windows client run the command telnet debiannas 12345 and paste the
contents of the key. Type Ctrl+] and then quit.
You should now be able to log in with ssh root@debiannas from your sofa.
Please test it. If you’re on your desktop testing in Hyper-V you can close the
window with the terminal/display, you won’t need it anymore (probably).
# The primary network interface
allow-hotplug eth0
iface eth0 inet static
address 10.9.8.145/24
gateway 10.9.8.1
Disabling SSH password auth
Once logged in as root edit the sshd configuration using nano /etc/ssh/sshd_config. Use Ctrl+W to search for PasswordAuth and remove #
to uncomment the line and make sure it now says: PasswordAuthentication no.
Type Ctrl+X and save the file when prompted and run systemctl reload sshd
for new config to take effect.
Give root a public key (you’ll only be logging in to admin this server)
We’re going to reuse the same key for the root account (because we can and it’ll
make our life easier).
on the server run:
$ su -c 'mkdir /root/.ssh'
$ su -c 'cp .ssh/authorized_keys /root/.ssh/authorized_keys`
Install dropbear-initramfs to use SSH unlock root remotely
as root:
# apt install dropbear-initramfs
# cp /root/.ssh/authorized_keys /etc/dropbear-initramfs/authorized_keys
# echo 'DROPBEAR_OPTIONS="-FEsjk -c /bin/cryptroot-unlock"' > /etc/dropbear-initramfs/config
# update-initramfs -k all -u
Test it by rebooting, (e.g. systemctl reboot), and try to login ssh root@debiannas
Note: You should probably make sure your server for unlock purposes always has
the same ip address, in case you have port forwarding setups or in case it’s
hard to figure out how to reach this host.
# echo 'IP="${ip_address}::${gateway_ip}:${netmask}:${optional_fqdn}:${interface_name}:none"' \
> /etc/initramfs-tools/conf.d/ip
# update-initramfs -k all -u
You can temporarily disable StrictHostKeyChecking temporarily:
PS > ssh -o StrictHostKeyChecking=no root@debiannas
… or even better, you can use this opportunity to save the hostkey on your
client:
PS > ssh-keyscan.exe -t ecdsa debiannas | Out-File FilePath = .\.ssh\known_hosts -Append -Encoding utf-8
$ ssh-keyscan -t ecdsa debiannas >> ~/.ssh/known_hosts
Securing the server / changing the keys/passwords
It’s possible this is not your first time settings this up, and you’ve used
secure passphrases and already have secure ssh keys. If when you decide to
change your keys/password here’s how.
Removing the temporary non-root user and group
$ deluser --remove-home testuser1
$ delgroup testuser1
Updating from stable to testing
This turns your Debian system into more of a “rolling release” kind of
distribution. You can update components to get current stuff when you want to,
and not just get bugfixes until, once in a while a new major release of
everything is out, and you have to deal with everything. As of January 2021, you
should probably do this before setting any fancy mount options.
The procedure is detailed here: https://wiki.debian.org/DebianTesting .
In the end your entire /etc/apt/sources.list file should look like:
deb http://deb.debian.org/debian/ testing main
deb-src http://deb.debian.org/debian/ testing main
Then, it’s a simple matter of issuing:
# apt update
# apt full-upgrade
You might need to answer a few questions along the way, e.g. regarding
sshd_config (which both we modified to change the PasswordAuthentication setting
and which also changed between versions).
You might have more than one kernel image at this point, perhaps an ancient one
and a recent one. That’s OK, eventually they’ll get autoremoved.
Setting useful filesystem mount options
Edit /etc/fstab, the first entry should be the one mounted into root “/”.
You might see entries pointing at /dev/mapper/sda3_crypt which make no sense
in cases when our filesystems move across devices.
Here’s a good set of defaults for a spinning disk.
Note As of January 2021, some of these require a somewhat new kernel,
unfortunately the current stable release ships with 4.19 that first appeared in
October of 2018, which is pretty old.
# <file system> <mount point> <type> <options> <dump> <pass>
UUID=381bd691-68fa-45f0-8703-cb926f732c88 / btrfs noatime,compress=zstd:3,space_cache,autodefrag 0 0
UUID=53f4b35e-e752-4dff-aa7a-a4f394f9a847 /boot btrfs noatime,compress=zstd:3,space_cache,autodefrag 0 0
UUID=F9FB-5A67 /boot/efi vfat umask=0077 0 1
Note: you can get UUIDs by running lsblk -f or using blkid.
After editing /etc/fstab you can reboot to get filesystems mounted with new
options, or you can run:
# mount -o remount /
# mount -o remount /boot
# grep btrfs /proc/mounts
Optionally, you can then proceed to explicitly compress the data on your
filesystems, and see what kind of sizing we have.
# apt install btrfs-compsize # tool to help us see stats about compressed usage
# btrfs fi defragment -r -czstd /
# btrfs fi defragment -r -czstd /boot
# not all data will be compressed, but check
# compsize -x /
# compsize -x /boot
Not all data will be compressed, but for root “/”, you may see something like:
root@debiannas:~# compsize -x /
Processed 26624 files, 18852 regular extents (18884 refs), 12931 inline.
Type Perc Disk Usage Uncompressed Referenced
TOTAL 38% 410M 1.0G 1.0G
none 100% 68M 68M 68M
zstd 34% 341M 986M 990M
Removing root password
This will make it harder to login as root by accident in the future. You’ll
still be able to use Rescue mode, and you’ll still be able to login as root via SSH
Running usermod -L root will set the encrypted password to an impossible value
(starting with !), so that no password ever will match. The account will
remain enabled otherwise, for example, it’ll have a default shell and so on.
This means that from now on, you’ll definitely need to enter Rescue mode to do
things.
Changing the disk passphrase
We’re using a password/passphrase to protect the much larger “volume encryption
keys” keys stored in the LUKS header of the raw volume. We can change this
passphrase without re-encrypting everything - and we probably should. Changing
the volume key is also possible, but we won’t do it here.
In fact, we can have several keys or passphrases if we so wish - we’ll
demonstrate that:
- Dump some information about our encrypted volume
- Add a new passphrase
- Dump some information again and notice an additional keyslot
- Remove the old passphrase
- Dump information again and notice we only have keyslot 1
# dump information about our volume
cryptsetup luksDump /dev/sda3
root@debiannas:~# cryptsetup luksDump /dev/sda3
LUKS header information
Version: 2
Epoch: 3
Metadata area: 16384 [bytes]
Keyslots area: 16744448 [bytes]
UUID: 176e7f71-49bd-465c-8a52-c4dff89e8856
Label: (no label)
Subsystem: (no subsystem)
Flags: (no flags)
Data segments:
0: crypt
offset: 16777216 [bytes]
length: (whole device)
cipher: aes-xts-plain64
sector: 512 [bytes]
Keyslots:
0: luks2
Key: 512 bits
Priority: normal
Cipher: aes-xts-plain64
Cipher key: 512 bits
PBKDF: argon2i
Time cost: 4
Memory: 430747
Threads: 1
Salt: 97 60 44 94 8e a5 5c 03 ee 6f eb 0e 33 1d f0 85
04 62 c7 1d 2f a2 64 c7 53 21 90 08 5d 1a 96 e3
AF stripes: 4000
AF hash: sha256
Area offset:32768 [bytes]
Area length:258048 [bytes]
Digest ID: 0
Tokens:
Digests:
0: pbkdf2
Hash: sha256
Iterations: 91022
Salt: 94 0a 3e a3 7a 5d 91 d9 dc ef 68 4f eb bd f7 98
29 ee 2a d8 48 8c 60 14 ed 69 e0 4e d4 45 bc 86
Digest: 0a df 75 01 0a 19 99 bc 3b ac 25 fd 0b e5 fc 16
d4 6f a4 87 55 e8 b2 11 2b 14 13 10 f4 30 8a d4
root@debiannas:~# cryptsetup luksAddKey -y /dev/sda3
Enter any existing passphrase:
Verify passphrase:
Enter new passphrase for key slot:
Verify passphrase:
root@debiannas:~# cryptsetup luksDump /dev/sda3
LUKS header information
Version: 2
Epoch: 4
Metadata area: 16384 [bytes]
Keyslots area: 16744448 [bytes]
UUID: 176e7f71-49bd-465c-8a52-c4dff89e8856
Label: (no label)
Subsystem: (no subsystem)
Flags: (no flags)
Data segments:
0: crypt
offset: 16777216 [bytes]
length: (whole device)
cipher: aes-xts-plain64
sector: 512 [bytes]
Keyslots:
0: luks2
Key: 512 bits
Priority: normal
Cipher: aes-xts-plain64
Cipher key: 512 bits
PBKDF: argon2i
Time cost: 4
Memory: 430747
Threads: 1
Salt: 97 60 44 94 8e a5 5c 03 ee 6f eb 0e 33 1d f0 85
04 62 c7 1d 2f a2 64 c7 53 21 90 08 5d 1a 96 e3
AF stripes: 4000
AF hash: sha256
Area offset:32768 [bytes]
Area length:258048 [bytes]
Digest ID: 0
1: luks2
Key: 512 bits
Priority: normal
Cipher: aes-xts-plain64
Cipher key: 512 bits
PBKDF: argon2i
Time cost: 4
Memory: 456323
Threads: 1
Salt: d7 aa 82 a7 77 7f d3 52 a8 e9 e6 33 ed 56 05 13
14 50 15 3a 8d 49 2b 33 c3 91 64 c3 3c c7 dd 95
AF stripes: 4000
AF hash: sha256
Area offset:290816 [bytes]
Area length:258048 [bytes]
Digest ID: 0
Tokens:
Digests:
0: pbkdf2
Hash: sha256
Iterations: 91022
Salt: 94 0a 3e a3 7a 5d 91 d9 dc ef 68 4f eb bd f7 98
29 ee 2a d8 48 8c 60 14 ed 69 e0 4e d4 45 bc 86
Digest: 0a df 75 01 0a 19 99 bc 3b ac 25 fd 0b e5 fc 16
d4 6f a4 87 55 e8 b2 11 2b 14 13 10 f4 30 8a d4
root@debiannas:~# cryptsetup luksRemoveKey /dev/sda3
Enter passphrase to be deleted:
root@debiannas:~# cryptsetup luksDump /dev/sda3
LUKS header information
Version: 2
Epoch: 5
Metadata area: 16384 [bytes]
Keyslots area: 16744448 [bytes]
UUID: 176e7f71-49bd-465c-8a52-c4dff89e8856
Label: (no label)
Subsystem: (no subsystem)
Flags: (no flags)
Data segments:
0: crypt
offset: 16777216 [bytes]
length: (whole device)
cipher: aes-xts-plain64
sector: 512 [bytes]
Keyslots:
1: luks2
Key: 512 bits
Priority: normal
Cipher: aes-xts-plain64
Cipher key: 512 bits
PBKDF: argon2i
Time cost: 4
Memory: 456323
Threads: 1
Salt: d7 aa 82 a7 77 7f d3 52 a8 e9 e6 33 ed 56 05 13
14 50 15 3a 8d 49 2b 33 c3 91 64 c3 3c c7 dd 95
AF stripes: 4000
AF hash: sha256
Area offset:290816 [bytes]
Area length:258048 [bytes]
Digest ID: 0
Tokens:
Digests:
0: pbkdf2
Hash: sha256
Iterations: 91022
Salt: 94 0a 3e a3 7a 5d 91 d9 dc ef 68 4f eb bd f7 98
29 ee 2a d8 48 8c 60 14 ed 69 e0 4e d4 45 bc 86
Digest: 0a df 75 01 0a 19 99 bc 3b ac 25 fd 0b e5 fc 16
d4 6f a4 87 55 e8 b2 11 2b 14 13 10 f4 30 8a d4
root@debiannas:~#
You can test the new passphrase with cryptsetup open --test-passphrase /dev/sda3.
We can also use the cryptsetup tool to shrink and enlarge partitions if/when
needed.
Adding more disks
We’re assuming we’re adding our second disk for redundancy and safety.
We’ll assume we’ll be adding a third / fourth / fifth /… disks in some kind of
raid mode that combines capacity and safety for our main data and root
partition. Boot should be as reliable as possible, so we’ll just be using raid1.
For testing /practicing in a VM / Hyper-V we’re attaching dynamically expanding
disks of the same size.
As part of “How to use btrfs raid5
succesfully”,
the number of drives should always be one higher than minimally required by the
raid level - with the exception being when you have only 2 disks. (Background
TL;DR if a disk is gone, you’ll want your array to be writeable to make it easy
to repair - ie. you’ll need to have a minimum of disks left with 1 disk missing)
Using 16T disks:
| disks | Metadata | Data | Capacity | Efficiency | Redundancy |
|---|---|---|---|---|---|
| 1 | dup | single | 16T | 100% (x1) | none (backup only) |
| 2 | raid1 | raid1 | 16T | 50% (x2) | 1 disk (downtime) |
| 3 | raid1 | raid1 | 24T | 50% (x2) | 1 disk |
| 4 | raid1c3 | raid5 | 48T | 75% (x1.33) | 1 disk |
| 5 | raid1c3 | raid6 | 64T | 80% (x1.25) | 2 disks |
| 6 | raid1c4 | raid6 | 80T | 83% (x1.2) | 2 disks |
Be aware that this glosses over the other practical aspects of running more
drives (vibration, failure rates of one of many drives, noise, cost of
motherboards/adapters/power supplies/cases/…),
Please do not use SMR drives for raid or or any “point writes” usage pattern.
If you use SSDs (SATA / NVMe) for write caching, you should first try to
understand the reliability math. Specifically that, while individual SSD drives
on average are more reliable than HDDs, their combination is not - in other
words with an HDD and SSD, should either of them fail, you’ve lost data on both.
Should you use SSDs for writeback caching, you’d do well to mirror them (beyond
the scope of this guide for now.)
In general, if you value your data, you should always have a backup (e.g. have a
second NAS or an rclone cloud solution. For home use, plan to test your restores
at least once a year)
Duplicating partitions
- Use sfdisk to dump the current layout into a file in script form:
# sfdisk --dump /dev/sda > sfdisk.script
- Use an editor to remove any UUIDs and device names from the script file (the UUIDs identifying partition types can stay):
label: gpt
unit: sectors
first-lba: 34
last-lba: 266338270
sector-size: 512
/dev/sda1 : start= 2048, size= 997376, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B
/dev/sda2 : start= 999424, size= 3905536, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4
/dev/sda3 : start= 4904960, size= 261431296, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4
- Feed the script to sfdisk to apply it onto new disk
# sfdisk /dev/sdb < sfdisk.script
- Verify it’s all good using
lsblk -fandblkid:
root@debiannas:~# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 127G 0 disk
├─sda1 8:1 0 487M 0 part /boot/efi
├─sda2 8:2 0 1.9G 0 part /boot
└─sda3 8:3 0 124.7G 0 part
└─sda3_crypt 254:0 0 124.6G 0 crypt /
sdb 8:16 0 127G 0 disk
├─sdb1 8:17 0 487M 0 part
├─sdb2 8:18 0 1.9G 0 part
└─sdb3 8:19 0 124.7G 0 part
sr0 11:0 1 336M 0 rom
root@debiannas:~# blkid
/dev/sda1: UUID="F9FB-5A67" BLOCK_SIZE="512" TYPE="vfat" PARTUUID="f40cc3a4-9912-4e3b-bb6e-3b6215954148"
/dev/sda2: UUID="53f4b35e-e752-4dff-aa7a-a4f394f9a847" UUID_SUB="426c16e1-9d43-4857-8079-9597a711c676" BLOCK_SIZE="4096" TYPE="btrfs" PARTUUID="2c8c3f91-bf63-466a-8bbb-21c1c26cd40d"
/dev/sda3: UUID="176e7f71-49bd-465c-8a52-c4dff89e8856" TYPE="crypto_LUKS" PARTUUID="3735d372-78d8-4cd9-9526-fd4a90c2ce03"
/dev/mapper/sda3_crypt: UUID="381bd691-68fa-45f0-8703-cb926f732c88" UUID_SUB="d6499a54-472f-4efd-8fff-dc5fcd86dcf2" BLOCK_SIZE="4096" TYPE="btrfs"
/dev/sdb1: PARTUUID="d7fa2a65-fa84-ee42-a0b5-f46a9d74263a"
/dev/sdb2: PARTUUID="fe13e5d9-524e-dc46-9f8c-d62a28fb641c"
/dev/sdb3: PARTUUID="6ee85f88-6449-4f4b-aefb-9e0b28b071c3"
Notice how PARTUUID are different between /dev/sda1 and /dev/sdb1.
Setting up the ESP partition as RAID 1
We can’t really convert the ESP partition to an mdraid partition, but we can
back it up, create the mdraid and restore the backup.
apt install --no-install-recommends mdadm dosfstools
tar -cavf /root/esp.backup.tar.gz -C /boot/efi . # archive our ESP
umount /boot/efi
# create the raid1 volume, use metadata=1.0 that keeps metadata at the end.
mdadm --create /dev/md0 --level=1 --raid-devices=2 --metadata=1.0 /dev/sda1 /dev/sdb1
# make a new / empty slighly smaller filesystem than before with same
# uuid/serial number as previoulsy, you can find it in /etc/fstab
mkfs.fat /dev/md0 -i F9FB5A67
mount /boot/efi
tar -xavf /root/esp.backup.tar.gz -C /boot/efi # unarchive our ESP contents
Adding additional /boot RAID 1 mirror
This is just a simple additional btrfs add device and balance operation to
convert chunks into raid1. We can do all of it online without unmounting
anything.
btrfs device usage /boot # shows current usage
/dev/sda2, ID: 1
Device size: 1.86GiB
Device slack: 0.00B
Data,single: 192.00MiB
Metadata,DUP: 190.62MiB
System,DUP: 16.00MiB
Unallocated: 1.47GiB
As you can see, by default we have system data and metadata duplicated twice on
the same device, but main data is held only once.
btrfs device add /dev/sdb2 /boot # add a device
btrfs device usage /boot
/dev/sda2, ID: 1
Device size: 1.86GiB
Device slack: 0.00B
Data,single: 192.00MiB
Metadata,DUP: 190.62MiB
System,DUP: 16.00MiB
Unallocated: 1.47GiB
/dev/sdb2, ID: 2
Device size: 1.86GiB
Device slack: 0.00B
Unallocated: 1.86GiB
We now have another device, but it’s unused. To actually start using the second
disk for raid1, we need to apply a balancing operation that will read all the
system, metadata, and data chunks and copy them across disks.
btrfs balance start -bg -mconvert=raid1 -dconvert=raid1 /boot
There’s very little data here, it’s just a kernel or two with initramfs image
data, even though balancing will proceed in the background while we can use our
system, chances are it’ll be done by the time you’re reading this.
If this were a much larger filesystem you can issue btrfs balance status to
check up on progress. You can even pause / resume. Shutting down / rebooting
half way through is safe (you can resume after rebooting.)
Addition additional encrypted raid1 root partition.
To do this, we need to create an encrypted volume from /dev/sdb3.
cryptsetup luksFormat --verify-password /dev/sdb3
Command above will ask you to confirm overwriting the data (even when the disk
is new and empty), and will then ask you to type thew new password twice. I’d
recommend using the same partition/password as that will cause the kernel
keyring to store it and you can unlock all your disks on this filesystem by just
endering the passphrase correctly once.
Formatting the volume will not open it. To do that you need to run:
cryptsetup open /dev/sdb3 sdb3_crypt`
The last parameter will create a /dev/mapper/sdb3_crypt name. To make this
/dev/mapper/… name permanent we need an entry in the /etc/crypttab file.
If you followed this guide, the file should contain only one line at the moment
looking like:
sda3_crypt UUID=176e7f71-49bd-465c-8a52-c4dff89e8856 none luks,discard
Parameters are described here: Debian Cryptsetup docs – index
You can change the file to look like:
sda3_crypt UUID=176e7f71-49bd-465c-8a52-c4dff89e8856 rootkey luks,discard,keyscript=decrypt_keyctl
sdb3_crypt UUID=7f090e5e-e6e2-43be-bc69-8ff5c51ef33c rootkey luks,discard,keyscript=decrypt_keyctl
The uuid is the same you get when you run blkid -o value -s UUID /dev/sdb3
Having decrypt_keyctl in options will store and use the passphrase in the
kernel cache. The rootkey is an identifier for the passphrase in that cache,
ie. devices with same passphrase should have the same value here.
To use the decrypt_keyctl you need to apt install --no-install-recommends keyutils, which will give you the keyctl command line utility.
In order for this crypttab to take effect on boot, you need to rebuild the the
initramfs image to contain this new /etc/crypttab file, by running
update-initramfs -k all -u.
After this, you now have an empty volume that is encrypted that you can add to
your btrfs root partition, similarly to how you added an additional empty
partition to /boot.
btrfs device add /dev/sdb3_crypt /
btrfs balance start --bg -mconvert=raid1 -dconvert=raid1
lsblk -f # should show the same filesystem uuid for both /boot and / filesystems
In the future, if you need to mount anything manually, using either the uuid, or
any of the decrypted block devices will mount the entire filesystem from as many
devices as are available at the time.
!!! Reboot and verify you’ve set everything correctly.
If you’re having issues mounting root, you can add break=premount to the
kernel commandline in grub to get dropped into a shell just prior to root
partitions getting mounted, which will allow you to issue cryptsetup and mount
commands manually.
Adding a third disk
// TODO
Example:
raid1c3 for system and metadata / raid5 for data
Installing Samba and backing up windows to it.
This is mostly interesting to windows users.
apt install --no-install-recommends samba
When asked about modifying smb.conf to read WINS data from DHCP, answer <No>.
WINS used with SMBv1 is old old old. We can use more modern DNS or ip addresses
instead, in fact, modern SMB protocol works fine with just TCP port 445 exposed.
UDP or other kinds of server discovery/negotiation are not strictly required.
Normally, windows servers would use WDS (Web Service Discovery ; UDP port 3072)
and not Avahi/Bonjour (Mac) or WINS (ancient) to discover nearby fileservers on
the network. In Windows 10 the old WINS / SMBv1 discovery is disabled by
default.
https://wiki.debian.org/Samba/ServerSimple
WSD
Monitoring
Admin nice-ties
tmux - to be able to leave operations running in the foreground for a long time,
or disconnect/reconnect and continue where you left off.
Updating and Upgrading software
apt update
apt full-upgrade
https://wiki.debian.org/UnattendedUpgrades
Automatic periodic maintenance - cron jobs
Once in a while, you may want to:
- check the integrity of your data on disks and verify checksums of rarely read
data to make sure it hasn’t bitrotted. - make snapshots and prune very old snapshots, so you can go back in time.
- backup your data from this nas, to another nas.
- update a mirror of your favorite linux distro.
- produce a report about the state of your server and email it to you.
Scrubbing
btrfs scrub start /
btrfs scrub status /
Snapshotting
Troubleshooting
Rescue mode
If you choose rescue mode, you’ll be able to go through a wizard that guides you
towards getting a shell.
On the boot menu presented by the netinst drive, choose “Advanced options…”
and then “Rescue mode…”. Answer a few questions about language and packages
your live environment need for rescue mode will begin to install. It’ll then ask
you for a hostname for networking and will ask you about timezones (why?), and
will then ask you for a passphrase to unlock the encrypted partition. (enter
it). It’ll then ask you what partitions to mount for you, pick
/dev/mapper/sda3_crypt, then choose Yes to mount the separate /boot and
choose Yes to mount separate /boot/efi. Then, in the “Rescue operations” menu,
choose “Execute a shell in …” and you’ll get a working root shell.
You can use this shell for all kinds of things, and you can type exit to go
back to the rescue menu to reboot.
Hyper-V keyboard
Keyboard works in linux, but for some reason the hyperv_keyboard module is not
included in initramfs image that the installer ends up building, so you won’t be
able to enter a password to unlock your root on boot. Let’s fix that. First,
follow the rescue mode intructions as above to chroot into your install. use
nano /etc/initramfs-tools/modules to edit that file, add a line at the end
that says hyperv_keyboard and press Ctrl+X to exit the editor (answer Yes when
prompted to save the file). Then run update-initramfs -u which should only
take a short moment.
You can now (optionally) type grub-install to re-deploy and bootloader
configuration. You can type exit and go back to rescue menu.
You can now reboot (you don’t need the DVD anymore, you can eject if boot order
is causing you issues), and go to First Login above.