The Ultimate Home Server - Herd of Netboot Raspberry Pi? Sure

on pfSense, do i set the dhcp option in:
TFTP server; or
Network booting > next server ?

thank you

option 66

Nice project, I have wanted to do this for a longtime but I only have Pi1 is it possible to do it on Pi 1?

1 Like

no, I think, only rpi3b+ and later (and that requies an update)

2 Likes

Yes ! I managed to get it working with a RPI3B+, truenas-scale as NFS/TFTP, and PFsense.
One thing I had to modify to get it working was to add this to the cmdline.txt

nfsroot=1.2.3.4:/nfsroot/rpi-pxe/rpi1,vers=3

When not specifying the NFS version, the RPI was not able to mount the NFS filesystem.

6 Likes

For the Pi1 and 2 the only option is to use berryboot, boot off the SD and then use an ISCSI provided storage:
https://www.berryterminal.com/doku.php/storing_your_files_on_a_synology_nas_using_iscsi

I am using this method with one PI1 and a couple Pi2/Pi3 since years now, even migrated the LUNs from a Synology to Truenas without issues …

1 Like

How difficult is it to change what distro the RPi is booting once it’s set up?

What if I wanted to switch from Raspbian to LibreELEC or RetroPie?

Also are there any theoretical security issues with this setup? Like how difficult would it be to hijack the boot to load a malicious OS.

Quite. For starters, the renegade should have full access to your TFTP server. At that point, you have a whole new set of priorities to worry about :roll_eyes: Assuming you have adequately secured your server from outside access, the rogue OS needs to know the exact data the RPi uses to boot from, otherwise said RPI will ignore any file presented to it.

1 Like

I guess what I was thinking is the case of someone trying to spoof the connection to the TFTP server since TFTP isn’t encrypted at all.

Yes, I understood. But TFTP is a local protocol, it shouldn’t be accessible from the web. If it is, you have a much bigger problem. Put it another way: if an outsider has access to your TFTP server, your entire network is completely insecure and everything is compromised.

1 Like

A question for people doing this with TrueNAS SCALE, what permissions are you running on your datasets for NFS shares for raspberry filesystems?

I’m in the process of trying out what works and so far the only thing that really works is the wide open ACL preset, when I try to lock things down a bit, I get the following complaints:

Unable to cd to '/home/pi'
[  OK  ] Stopped Getty on tty1.
[  OK  ] Started Getty on tty1.
         Stopping User Runtime Directory /run/user/1000...
[  OK  ] Stopped User Runtime Directory /run/user/1000.
[  OK  ] Removed slice User Slice of UID 1000

after which I get dumped back to login prompt.

Part 2 of this is coming featuring Ansible tips from @geerlingguy if I can talk him into it haha

O84HvSe

(He is really good!! also welcome!!)

13 Likes

I actually did… and just got to chapter 6 in the ansible book.

1 Like

One thing to note

Docker will allow containers to use NFS shares, however Docker itself will not run from an NFS share. Dockers default storage driver is overlay2 , which uses overlayfs , which as far as I know does not support NFS. An alternative would be to use an iSCSI share instead.

It’s a fairly important point considering the popularity of both Raspberry Pi’s & Docker are in the homelab.

The question is, could you use the zfs filesystem driver with Docker on an NFS share?

2 Likes

Heh, thanks! I’ve been meaning to cover netboot in a video someday (and to convert all my racked up Pis to it, so I can ditch their hard-to-access microSD cards), but until then, I’ll be pointing people at this forum post. Thanks for posting all the details!

14 Likes

@geerlingguy covers the new beta of Network Install:

5 Likes

Yeah, the inclusion of pi 4 images is nice, I still have my original self-compiled image running and it’s still going strong. And Void has a lot of packages in the repo. But, for the Pi, especially if you run one with <2GB of RAM, I would highly recommend Alpine if it fits your needs.

Heck, even with maintaining a bunch of SD cards, Alpine Linux on the Pi is running by default in Diskless Mode (“frugal mode”), which reads the image from an SD card (or storage of choice) and then loads the whole system onto RAM. If you want, you can unplug the SD card after the boot is over and it will be fine, so long as you don’t have any partition from the SD still mounted (only for safety of the SD, the system will still be fine without it, even forcefully unplugged).

I’m running Alpine on my pi 3 from an ancient 2 GB microSD card. Whenever I do updates, I just do a lbu commit and the directories I have set to get backed up are stored on the SD card. I can then install whatever crap I want, but after a reboot, it will revert to the previous state from the last commit and not have anything else installed, as those were only installed in RAM and not committed to disk. That way, I get basically 0 read or writes on the SD card besides when doing an update and a lbu commit.

What lbu commit does is creating a tar archive on whatever mount point you give it. It doesn’t have to be on the SD card, you can boot a completely generic Alpine image via PXE and have Alpine get its own tar.gz archive to extract on its RAM FS. This way, you even cut the network cable (so long as you don’t use PoE obviously) and you can eliminate network read/writes, you don’t need to have your / partition mounted on a NFS server.

Jeff is a L1T Forum member? Noice.

@geerlingguy @rustyb0y welcome to the forum, both of you. A bit late on my part, but nonetheless, welcome! Hope you’ll feel like home here, we have a lot of interesting topics that you can follow, from Wendell’s posts, to our forum members blogs featuring anything from server stuff, audio stuff, or pop culture things. Here’s another post in which I mention the most prominent blogs on the forum:

1 Like

then perhaps the thing to do is back the iscsi lun on the fileserver as a zfs block device for the actual docker containers. to appear to the rpi as a natively formatted partition (onto which that can be overlayfs). so a really good question is what is the most effecient and performant filesystem to put onto that zfs backed iscsi lun, once it also has a working overlayfs sitting ontop. for your docker build times etc. thats an interesting thing to benchmark.

moving on. if you bind mount specific folders into the containers. for actual data, those can be zfs folders or datasets. which can then be doubly shared over nfs

you also want to be able to re-create the docker containers should the monolithic partition become bad or messed up. so if it includes docker volumes which are not bind mounted then those need to be either recreatable from an immutable template. or regularly backed up elsewhere. then restored… hence the suggestion of bind mounted folders

perhaps a more elaborate and interesting solution is kubernetes. but that would be much more complex to configure and setup. also its hard to get extra rpis right now for a cluster. unless they are some other different alternative device

Agreed, I generally advice new admins, who is looking into learning - to consider recycling a old laptop and/or desktop that they can ask around for “free” (especially netbooks)

Unless they are space (ie. living in a dorm), or energy constraint - most “do X on PI” tutorials, can be replaced by a general computer - and is much cheaper, and less intimidating to experiment on a “PC” for newcomers.

PI, to me should primarily be used in IOT use case. For “server” workload use case, its more of for rule of cool, and youtube keyword optimisation. (nothing against jeff, its cool)

Nice write up, picked up a bunch of dell Wyse 3040’s and netbooted them using parts of this guide as a reference. I used the MAC instead of getting the output of vcgencmd.
Is there a simple way to automatically update the tftp server to point to a new kernel after upgrading it? I know apt has a post upgrade routine I can add a bit into, or just have it in my bashrc to check every restart and update the pxelinux file with the new kernel.
Just wondering if there’s a quicker a neater solution that anyone has used?

1 Like