Total noob to enterprise gear, so excuse me if this one’s obvious, but I’ve picked up an HP ProLiant DL360 Gen9 Server to experiment with and to start building a backup and file share server for home. Looking inside, I didn’t notice a place to put an NVMe drive (which I think this one may have been before NVMe become used for storage very often) but noticed it has a micro SD card slot as well as 2 stacked USB ports inside, which I figured to be intended to be the intended boot device. I was wondering at the time about how durable a USB stick or SD card would be (this one lasted most of a year) though I figured it wouldn’t be anywhere near what an NVMe drive would be. …so that still leaves me with a question that I wasn’t totally sure of then, but even less now. How are most enterprise servers intended to be booted for normal operation? Is it supposed to be ran off the USB or SD card, and I occasionally dd the boot device to another device that is kept at the ready, or one of the removable hard disks? …and if it’s supposed to boot off the SD card or USB thumb drive, what sort of device is intended to go in it, or would be durable enough to hold up to running an OS off of?
Depending on backplane, the front drive bays are SAS, PCIe (NVME) or Tri-Mode.
Those can be used for boot, more often they are used for USB-license dongles.
They normally do a dedicated raid 1 with ssds. Some servers even have 2 dedicated 2.5 bays in the rear for the OS ssds. However the mirco sd and usb ports can also be used. But i wouldn’t use them for a full operating system like windows server as the read and writes will probably kill a micro sd or usb drive. truemas and proxmox or vmware or other hypervisors would work on the usb port/micro sd pretty well.
If you wanted to do an nvme. You can always use a pcie adapter if you have an open pcie slot available. I would probably do that with an optane or similar high endurance nvme. depending on how much space your OS needs. I run a 32gb optane M10 for truenas.
IMO, after running home servers for nearly 30 years, the answer is basically this in order of priorities:
Priority 0 - PLAN FOR FAILURE!!! Plan on the OS disk dying. No amount of monitoring nor warning indicators will prepare for the sudden and immediate dying of the disk. It will happen. So plan accordingly on what you’ll do. Don’t store critical files or keys or anything on it.
Priority 1 - Mirror disks of two identical disks, and hopefully purchased at two different times to get two different lots. OS mirroring is superior here these days, rather it be ZFS Mirror pair vdev, LVM, or alike.
Test that you can unplug one driver and boot every year or two.
Proxmox’s installation wizard has the best textbook example of ZFS Mirror OS drives - and it does it automagically during installation. Leverage that, and you can reuse the spare ZFS space for other long term storage like installation ISOs.
Priority 2 - High write-endurance disks. Like Samsung SM883 SATA drives. It’s best to change the OS to limit writes/logs/no swap, but high endurance disks should last 8-10 years.
Besides those two priorities, I would not invest into any more “fancy” features like “I want M.2 speeds” or Gen4 or whatever. Not worth it.
You don’t need P2. You’d just have to replace the drives a bit more often when they fail. And they WILL FAIL, without notice of any kind. You must PLAN on failure. So go with P1 immediately from the get go - so your family doesn’t get pissed at the server being down, again.
Note: If all you got are two M.2 slots, that works too, but high endurance M2 drives will be much more expensive - for basically being idle for years and years.
Back in the day the internal USB or CF slots could be use to load the hypervisor. In a cluster, loosing one node due to the CF failing was not that big of a deal. Now days, normally 2 cheap SSDs are used.
For my home servers, I pickup a small used enterprise SSD from Server Parts Deal. I can rebuild Proxmox quickly if it fails. Or I use 2 in RAID 1.
NVMe to PCIe card will work in that server, but it might not have the option to boot from it.
The OS on the server is of no real importance, doing backups of the VM’s on the server is.
Don’t bother mirroring boot drives and the server OS, if it goes boom a usb stick and 10 minutes will get you right back where you started from. The valuable parts are your VM’s, that’s why you do backups of those.
If uptime of your VM’s are important, have more than one server and set up some high availability for the VM’s. Again, the server OS itself is not important, if it goes boom you pop in a bootable usb and are back up and running 10 minutes later.
I did this for a few years and after the 2nd USB stick failure, I’d never do it again.
Pissed off family too many times with the daughter, “again?!” when I’m sitting in Russia on client site and the family has to wait a few weeks until I return and “find the time.”
Plus, Proxmox doesn’t support USB stick nor eMMC - wirhout edits (I posted a hack on how to do that)
Mirror of two cheap SATA SSDs > USB stick.
I guess I was unclear, I’m not advocating using a usb stick as your boot drive, I sure wouldn’t.
The point I was trying to make was that reinstalling for example proxmox from a usb stick onto an ssd takes all about 10 minutes, so no need to be too fuzzy about whether that installation is on mirrored drives or not.
Ah, sorry about that! Well, the advice is still valid. Router-on-a-stick thingy. ![]()
To a network guy, router-on-a-stick is something completely different.. ![]()
As for keeping the family happy I recommend not having them dependant on the lab part of a homelab.
My personal setup that isn’t my homelab is two low power file servers, essentially nas boxes build with standard parts. The main nas has 4 drives, the backup one has 5. I run ubuntu server on both, no raid, but set up with mergerfs pools.
Every night at 3 am the 4 drive nas rsyncs onto the 5 drive nas. If I accidentally deletes a file on the main nas, I have a backup on the second one.
Family uses the 4 bay nas. If something goes wrong with the 4 bay nas, I’ll just change the dns record to point to the 5 bay nas and everything keeps on running until I can fix the 4 bay one.