After many years of grinding by with an insanely underpowered Synology NAS as my storage server, I have finally acquired all the necessary hardware to build my own FreeNAS server. Only with the help of a free Xeon E5645 and 24GB of unbuffered ECC memory from work.
Specs:
Motherboard: EVGA X58 FTW3
CPU: Xeon E5645
GPU: Some old thing just for video output
Storage: 6x WD Red 3TB (intended for Raid-Z2)
1x 8GB USB stick (bootable FreeNAS installer)
2x 8GB USB stick (FreeNAS install location)
This motherboard has given me nothing but grief so far. First it took me 3-4 hours of troubleshooting and researching to figure out that the mobo has an idiotic limit of 8GB for bootable USB sticks. Goodbye freshly bought 2x 32GB USBs and multiple boot environments
That aside, I have absolutely no problems installing FreeNAS to the 2x 8GB USBs . That works perfectly and I made sure to select the “Boot via BIOS” option (NOT “Boot via UEFI”) during the install seeing as I have a much older motherboard (no UEFI). Unfortunately the FreeNAS installer creates GPT partitions on the dual USBs, which my mobo is incapable of booting from. If either or both of the FreeNAS install USBs are plugged in, the system gets stuck at the POST screen with a POST-code of 50, meaning “USB Initialization”.
I have read a bit about flagging the partition as “boot” instead of “grub_boot” with the help of a GParted-Live bootable USB, which I was not able to do on either of the USBs with the fresh FreeNAS install.
Elsewhere I read something about creating a hybrid MBR partition, but gdisk (GPT fdisk?) in windows constantly has errors reading the USBs.
At the moment I am at a serious loss and any help or nudges in the right direction would be greatly appreciated.
Thanks in advance!
Jared
EDIT: The versions of FreeNAS I tried installing were 11.1-U6 and 11.2-RC1. Both with the same outcome. Both times the bootable USB was created with Rufus with the partition scheme set to “MBR” and target system to “BIOS (or UEFI-CSM”. Both the ISO and DD creation options didn’t make a difference as far as I can tell.
I’m not a freebsd user, but … maybe create a separate GRUB bootable MBR USB disk which chainloads to the GPT disk created by freenas? Hopefully you will create a configuration that doesn’t need to be updated every time you update freenas. I figure I could make this work for linux, but no idea for freebsd/nas.
Yes, but again I don’t know enough about freebsd boot process to comment too much.
When you are looking at grub chainloading make sure you aren’t confused by grub < 1.0 and grub > 2.0, as the syntax and capabilities are different.
Would probably be easier to do a basic freebsd install to USB, and then work on building a chainloading bootable USB, than trying and do it with freenas initially. Or for a basic ubuntu install to USB and do the same, whatever is easier for you.
Ok the install script does write a PMBR, so it’s odd that your bios is getting hung up.
It will be a learning process, but you can manually partition one of the USB sticks using gpart in the installer shell, resilver the freenas-boot ZFS pool, and hopefully the system doesn’t depend too much on GPT…
Thanks for the feedback, guys! Much appreciated. @freqlabs When should I partition one of the USBs in the installer shell? Before or after I freshly install FreeNAS to said USB?
Another thing to mention: Is it weird than when I have a fresh install of FreeNAS and I plug that USB stick into my main computer (Windows 10 Pro), it shows up as two separate drives in windows explorer that I cannot open. Yet in disk management the usb shows up as “Healthy (Primary Partition) 7.47 GB”.
Could these be a further hint as to what may be causing my issues?
I was convinced it was just the X58 motherboard that can’t handle GPT. Was it from FreeNAS 9.3 onwards where the implemented GPT instead of MBR?
You’ll want to install to one USB stick, then use the installer shell to partition the other USB stick (the EXAMPLES section in the gpart man page linked above may be helpful). And then attach the new ZFS partition to your freenas-boot pool. It will resilver, then you can detach the first stick from the pool, partition it for MBR as well, reattach it to the pool and let it resilver again.
Next I partition the second USB with the freenas installer shell. What steps exactly? (Excuse my ignorance on the matter). Like this?
/sbin/gpart create -s MBR “device name”
/sbin/gpart add -t freebsd -s “my USB size” “device name”
/sbin/gpart set -a active -i 1 “device name”
/sbin/gpart bootcode -b /boot/boot0 “device name”
How do I add the second USB to the freenas-boot pool?
Does the resilvering occur automatically? How can FreeNAS know that the boot devices need to be mirrored?
If I partition the first USB for MBR, won’t the data on there be lost?
Disclaimer up front: Use at your own risk. I have not tried this, but it’s a fresh install so you should have nothing to lose (as long as you don’t accidentally wipe the wrong disks).
Yes
Try this (I will use da0 as the example):
# prep by mounting the freenas-boot pool of the freshly installed system
zpool import -N -f freenas-boot
mount -t zfs -o noatime freenas-boot/ROOT/default /tmp/data
# adapted from zfsboot(8)
gpart create -s mbr da0
# partition size rounded down to the nearest 16 MiB
fullsize=$(gpart show da0 | awk 'NR == 2 { print $2 }')
roundsize=$(( (fullsize >> 15) << 15 ))
gpart add -t freebsd -s $roundsize da0
gpart bootcode -b /tmp/data/boot/boot0 da0
gpart set -a active -i 1 da0
dd if=/dev/zero of=/dev/da0s1 count=2
dd if=/tmp/data/boot/zfsboot of=/dev/da0s1 count=1
dd if=/tmp/data/boot/zfsboot of=/dev/da0s1 iseek=1 oseek=1024
Say you installed to da1 and partitioned da0:
zpool attach freenas-boot da1p2 da0s1
# now da1 will be resilvering, observed with
zpool status freenas-boot
# when that completes:
zpool detach freenas-boot da1p2
gpart destroy -F da1
# then partition as above, but don't recompute roundsize
gpart create -s mbr da1
gpart add -t freenas -s $roundsize da1
gpart bootcode -b /tmp/data/boot/boot0 da1
gpart set -a active -i 1 da1
dd if=/dev/zero of=/dev/da1s1 count=2
dd if=/tmp/data/boot/zfsboot of=/dev/da1s1 count=1
dd if=/tmp/data/boot/zfsboot of=/dev/da1s1 iseek=1 oseek=1024
# attach it back to the pool and let it resilver again:
zpool attach freenas-boot da0s1 da1s1
# after resilvering, finally
umount /tmp/data
zpool export freenas-boot
“automatically” in the sense that you just attached it to an existing vdev in the pool as a mirror so it automatically starts resilvering.
Yes you resilver twice. Good thing there is not much on the storage yet. ZFS only resilvers the used space, so at least you don’t have to wait for a whole lot of nothing to be copied
@freqnas You are an absolute GODSEND! Thank you so much for the time and effort.
There are a few hiccups for me though. The roundsize defining somehow results in a ‘0’ which is an invalid size for the partition. So I just used “gpart show” to get a rough estimate. The total is ‘15667…’ so I made the partition size ‘15660000’.
When I enter “gpart bootcode -b /boot/boot0 da1” (da1 is my device in this case) I get the error “gpart: /boot/boot0: No such file or directory”.
whoops yeah I forgot to subtract 9 from 24, since gpart show is in sectors (512 bytes, or 2^9)
And for the other problem ok, the installer doesn’t have that file on it but the installed system should, I think it’s mounted at /mnt/data/ so try /mnt/data/boot/boot0 if I remember right.
Alright. When I insert the USB with the fresh freenas install, how do I tell gpart to get it from said USB? Do I need to “mount” the drive with a command in the shell?
I took it out when I didin’t need it because I can’t get past the POST screen if it’s plugged in and I want to restart Hence why I plug it in only AFTER I’ve reached the FreeNAS installer shell. Otherwise there is no way of getting there.
Let me show you what I have with pictures:
I really don’t mind it if there are a few bugs. I JUST WANT TO BE ABLE TO BOOT INTO THE DAMN THING.
Exact same situation after install 11.2 to a USB. No boot/boot0 to be found anywhere.
If I can’t get it to work this week, I will use windows server or unraid to create a different kind of storage system. This is becoming absolutely ridiculous the amount of prep and work needed to just boot in only because one has an older mobo.
Hence why I plug it in only AFTER I’ve reached the FreeNAS installer shell. Otherwise there is no way of getting there.
