The Ultimate Home Server - Herd of Netboot Raspberry Pi? Sure

Let’s talk hardware for a sec.

Raspberry Pi all the things

OK, one concern I have is that everywhere you turn online is someone shoving a raspberry pi headlong into a project regardless of whether or not an RPi is the best fit for the problem you’re trying to solve.

However, for $35-$75 they are a good deal because they have a modest amount of compute and are friendly on power. They are a cheap and effective way to add whole-house audio or whole-house CO2 sensors.

There is a vibrant bazaar of raspberry pi devices that may be useful in the context of Ultimate Home Server/Knowledge Keeper:

I am using Raspberry Pis in my setup. Two things I try to do for quality-of-life

Netboot Raspberry Pi - Why?

Local storage, and managing local storage, when you have more than one raspberry Pi is a pain. There is no need. Since the RPi 3b+ it is possible to boot from the network if you have a storage server somewhere on your network. This makes it much easier to upgrade or replace hardware, to manage backups and ease-of-restore, and to scale out your network (more than one pi).

Why might you want a small army of raspberry pi in your house? Whole house audio, for one. But there are many, many whole-house audio packages/projects for raspberry pi and that’s not what we’re covering (yet).

This is all an important leadup to servers are cattle, not pets.

I get it. No one wants to be a sysadmin at home. Neither do I (okay, we both knew that’s not true, but… I get it). Having the RPi boot from the network is worth the setup headache at least because it makes the setup much more maintainable over a 5-10 year period. SD cards eventually wear out. When it dies, can you be up and running immediately?

If you want to share your experiences or setup you have, please do! I am sure everyone would love to see.

Raspberry Pi PoE

So the PoE had makes it so you can just plug your raspbery pi into a switch that provides power. One wire. This makes it easy to put your RPI in a wall, attic, or anywhere you could run a cat5e/cat6 network cable. The PoE hats out there don’t have a lot of spare power, though, so be careful if you add any USB devices to your RPi. I had trouble even using a z-wave USB dongle with mine because it drew too much power for the standard PoE hat. Symptoms of this malfunction included hard lockups.

Raspberry Pi devices aren’t really built for 99.999% uptime, either. With the PoE switch it is possible to write a script to reboot the port (by toggling power), too. (More on that later).

Understanding Netboot

Your Raspberry Pi may or may not be configured to boot from the network. Generally newer RPi 4 and 3B+ come configured to netboot but that wasn’t always true especially with the first 4/5 of the production runs of the 3B+. You will need to do an initial setup on an SD card, boot the pi, log into it, and configure it for netbooting. Sorry. 4 and later should be configured out-of-the-box to netboot without this initial setup.

Netbooting starts with your DHCP server. The DHCP server is what hands out IP addresses. On the Pi itself there is something that kickstarts a process called PXE booting. PXE stands for Pre-boot execution environment and, just like it sounds, it is meant to bootstrap the device, get a DHCP lease (IP address) and to get the configuration from the DHCP server as to where it can pull and boot from a bootable image.

The bootable image comes from a TFTP Boot server. TFTP is trivial file transfer protocol. It is very unforgiving of bad or flaky connections so if you have intermittent problems it may be a sign you have a bad cable somewhere.

TFTP just gives you the boot image and kernel. Once the system starts booting, it needs to mount the root file system. NFS, network filesystem, is just the thing. I found this writeup from 2020 with more info.

This guide also has more info:
Hack-a-day guide (older)

Another important note – the TFTP server has pi-specific configuration files. The RPi uses a file named for the serial # of the specific Pi in order to download configuration specific to that device. This is how it is possible for one boot image to many differen RPi devices.

Also, if you have a Synology nas (or even another nas with similar features), you can use it as the server that serves both the tftp image as well as NFS.

Of course if you have a Linux server or other server, you can use that too.

To be clear, it is not necessary to use the Synology as your DHCP server. PfSense and many other router distros support adding custom records to the DHCP server for netbooting.

You might also want a setup for your home network that allows machines other than your Raspberry Pi machines to boot from the network. It is useful for whole-machine backup imaging, bare metal backup/restore, and much more.

I expect to cover more of that kind of thing in this series.

So, in a nutshell the process is

PXE (on the RPI) > Query your DHCP server > DHCP record points to TFTP server > TFTP boot image > TFTP Rpi config (serial # specific) > nfs mount / based on that config.

The Actual How To

I think it is very important that the reader understand what we are doing before we start copy-pasting commands. This guide is a great starting point if you understand the moving parts.

The tutorials talk about updating eeprom. I didn’t need to do that on mine. Newer RPi probably won’t need it, but if yours isn’t booting or trying to boot, check the eeprom settings stuff.

raspi-config has a ‘boot order’ option that accomplishes the same thing:

… but if you don’t see that or can’t set network boot order THEN you might need to update your RPi eeprom! which is tedious.

Still working on this

  1. Setup TFTP Server and NFS server (empty folders) (add nfs share for tftp boot folder and root file system)
  2. DHCP server set option 66 , add a reservation for the mac address(es) of the Pi to the dhcp server
  3. configure the nfs server allow read/write only from the IP address(es)
  4. get the raspberry pi serial # and make a directory tftpboot/rpi serial
  5. use rsync to copy /boot to the tftp boot folder; use rsync to copy / to the root fs nfs share

I created two shares:




rpi-tftpboot contains directories – one for each pi that is the serial (output of the command vcgencmd otp_dump | grep 28: | sed s/.*://g )

e.g. rpi-tftpboot/f00f0021

and rpi-pxe contains the hostname of each of my pis. so the first one is rpi1, rpi2 and so on.

I created /nfs back on the raspberry pi (booted from sd card) and mounted:

/nfs/boot > nfs-server/rpi-tftpboot/f00f0021
/nfs/root > nfs-server/rpi-pxe/rpi1

first rsync /boot/* from the running pi to /nfs/boot
rsync / --exlude /nfs / to /nfs/root

It is necessary to then modify cmdline.txt in /nfs/boot to be something like:

console=serial0,115200 console=tty1 root=/dev/nfs nfsroot= rw ip=dhcp elevator=deadline rootwait

and further to modify /nfs/root/etc/fstab to be something like:

proc            /proc           proc    defaults          0       0
#PARTUUID=f6242238-01  /boot           vfat    defaults          0       2
#PARTUUID=f6242238-02  /               ext4    defaults,noatime  0       1  /boot           nfs    defaults          0       2  /               nfs    defaults,noatime  0       1
# a swapfile is not a swap partition, no line here

If you have a synology or other nas, you can do the nfs/tftp setup right through the gui.

In my case I only had to set DHCP option 66 with the IP of the tftp server.

The tftp boot process is somewhat complex. Other tutorials talk about un-gzipping vmlinuz or setting eeprom options or setting other dhcp options. In 2022, I don’t believe any of those extra steps are necessary.

It is necessary to run raspi-config and go to advanced > boot order and enable network booting, proably. If you plug in your RPI but it seems to be stuck and not doing anything on your network with HDMI plugged in port 0, but no SD card, then it is likely either a DHCP option that is missing or you need to run raspi-config and pick netboot.

Froom here if you need to netboot other RPi, all you need to know is their serial #. I recommend setting a fixed IP via DHCP and only allowing NFS write from that specific IP. If you do those two things all that’s necessary is to copy the boot folder and root folder to new names and then your other raspberry pi should be good to go.

Of course it is a best practice to enable OS updates on those and none of that really goes away with this. All we’ve done, so far, is eliminate the need for an SD card with your Raspberry Pi. (Allthough you might need one initially to copy the /boot and root partitions, and to configure booting from network…)

Some Extra Steps for Truenas 13 and later:

A user wrote in and we did some troubleshooting:

On my install I can not check the ‘All dirs’ check box if there is more than one path listed and to complicate matters further, I cannot create two shares with the same root path, TureNAS tells me they must be separate datasets.

Working solution – TrueNAS 13.0 BETA1

Instead of dropping to shell and creating the require directories using mkdir

Create dataset ‘nfs’ on storage pool ‘tank’

Create sub-dataset ‘rpi-pxe’ under ‘nfs’

Create sub-dataset ‘rpi-tftpboot’ under ‘nfs’

Create individual NFS shares of both rpi-pxe and rpi-tftpboot, set ‘Maproot User: root, Maproot Group: wheel and tick the ‘All dirs’ checkbox

Everything else should be fine.


Also, the OrangePi running Armbian makes a good choice for hardware. The price got less competitive for qty one but buying a stack for your next multi-purpose ARM mini rack (I need to finish that…) defray’s the shipping.

I’ve got the utmost respect for the Armbian/Sun-Xi project people. Reverse engineering the non-GPL compliant AllWinners was and is amazingly good work.


Does it net boot? Not managing is on an array of sd cards is important

1 Like

They’re MMC based. Although I guess some models probably have an SDCard available.

Ya’ know, that is a good question though. I can’t recall if they’re using GRUB to bootstrap and I didn’t have reason to netboot specifically. I will have to fire the rack up and see.

1 Like

I learned lots from the LTSP, way back in 2008-ish. Including the (then) undocumented feature of serving non-native-arch images: I used 64 bit Debian host to create an i386 vmlinux setup. It worked, but the use case I had disappeared unfortunately.

I used a dual Gbit port mainboard (AM2 or AM3, can’t recall) with one port as gateway/router/DHCP+NFS server for the LTSP side, the other was a DHCP client on my regular network. Webmin was used to config and manage the network stuff (NFS included). Unfortunately, network port assignment was a bit flaky so quite often a reboot caused the wrong port being assigned the wrong task. With the use case gone I quickly gave up on the project so I never figured that out permanently. I’m assuming things have improved since. :slight_smile:

I was an LTSP nerd back in the day also. I setup a [email protected] farm on it, and also installed a distributed C compiler for speeding up package installs in Gentoo.

It was a mish-mash of spare parts so all the nodes were different from each other in some way. One day I found out I could speed up the whole cluster by removing one node that was slowing everything else down. Makes perfect sense now, but back then… :exploding_head:

Thanks Wendell for posting this. I’ve had something very similar in mind for quite some time now.

1 Like

The orangepi zero comes with SPI flash onboard that can hold pxe uboot.

Someone wrote some instructions for orange pi zero netbooting here.

I’ve been hearing complaints from my friends who work at Facebook about PXE being commonly broken on IPv6 only networks.

I guess “rolling your own uboot” is one way to get around it. (Facebook does their own BMCs using golang on bare metal… at least they did last I heard)

How does raspberry pi stack up? Does it netboot on IPv4-less networks?

Also, microsd storage is free these days. Kioxia Exceria 16GB can be found for $3 in retail + $3 shipping.

That’s plenty for a bootloader :slight_smile:

The annoying thing is reliability. (not specific to Kioxia - the CPU inside of the microsd card is typically some 100MHz arm M0 with not too much ram, that can’t do lots of fancy FTL stuff, and the microsd standard is also limiting).

As long as you can swap them…


I have a question regarding the boot order. I have an Intel X550-T2 NIC that does up to two times 10Gbits/s over RJ45. I am currently in the UEFI menu looking at the option Legacy Boot Protocol. It features the options 1. None 2. PXE 3. iSCSI Primary 4. ISCSI Secondary 5. FCoE.
The option featuring iSCSI Primary seems to be there to boot an operating system from a SAN! @wendell do you have experience with this, would it possibly as easy as: SCSI Primary → Booting

1 Like

Yep. Works fine. os needs support. but yep it’s fine


hey wendell. i installed void linux 64bit on my rpi4. since they have recently been making builds. works well and seems llike a clean and stable option with few bloat. easy to keep updated and also the package manager is like a ports tree system is good for making packages with. of course maybe it lacks other stuff that rapbian has. so maybe that is a tradeoff depending what you are doing with it

will be looking forwards to trying pxe boot in the future. thanks for this guide

1 Like

BTW a long time ago i created a pxe docker image. which included the necessary ipxe and tftp server and dhcp proxy erver etc. all togther. but it wasnt made specifically for booting rpis with. good idea though :grinning:

Gosh how much i wanted this guide for too darn long.

I been too used at work in making sure equipment was designed to last X years+, that I been holding back on the pi’s on my home network automation stuff for ages.

I may had only 2 SD card failure in the past with some pi’s, but the problem was it always by jinx luck, fail at the “worse time possible” (in the middle of personal/work emergency). This helps me mitigate that.

1 Like

where can you still buy Pi’s for near MSRP prices?

1 Like

I plan on testing out this guide soon because one of my biggest pet peeves is not having an easy way to do deeper editing of a RPi remotely.

EDIT: Is there a way for the overly paranoid to secure this method or would you consider this method already secure?

I agree. For example, sometimes I feel like the Pi is actually too much machine and too expensive for some of the smaller projects even things like that home audio setup.

What I think might be interesting is looking into a unified framework for using ESP32s (or similar) for various IoT tasks.

They can be had for much cheaper than a Pi and less power hungry but without giving up Wifi or Bluetooth.

There you go …

Somewhat steep learning curve, but supports commercial modules like these:

I use two esp3286 based ones to control sump pumps and pool pumps, and an atmel based one to control gates and remotely hard reboot a couple raspberries. I do use raspberries to collect sensor data from an rs485 network and to control my heat generator … I agree that the raspberries are overkill, and I also agree that for mission critical (e.g, the wife needs to use it) type of applications raspberries may be too unreliable as you never know when they will crap on you… While the microcontrollers are harder to setup but much more reliable. Also, when connecting electronics to control 220v mains I prefer to go with industrial products as opposed to rolling out my own…

1 Like

So is it possible to do netboot via wifi? That being either built in wifi or via some sort of extra hardware. I ask mainly since there are places I could place these but wiring is not an option currently or in the near future.

1 Like

PXE boot via wifi is not supported, you could in theory do it with Berryboot, but it’s going to be unstable. Once your root FS is on a network share or an ISCSI LUN any interruption between the PI and the storage will cause unpredictale behaviours, from small interruptions in whatever the pi is doing to the root fs going read only to the pi completely hanging and requiring a hard reboot. Achieving stability of the network over the Pi wifi is going to be challenging …


yeah that’s what I thought

1 Like

On a separate note, I think a lot of the SBCs can do netboot. Here is one from RockPro64. I still havent tested it yet:


Most of the other SBCs have a SPI partition or switch which enables running a custom boot loader. I am biased, but if you are not going to use a Pi, I would recommend something with an AmLogic chip for a RockChip since those tend to get the best aftermarket support.

I use combo of Odroids and RPis in my setup. I don’t actually net boot mine.