I am not a beginner to Linux, but I am a beginner to self-hosting, and I fell down the rabbit hole quite a bit, to be honest
My current setup is as follows:
Dell Optiplex 7050 micro (16GB DDR4, 500GB NVMe)
4TB WD Red (connected via USB to the Optiplex)
Proxmox on the host, VMs inside
The future plan is to implement a ZFS pool for the NAS, since the single 4TB drive holds a bunch of data I wouldnāt want to lose. My idea is to get some dock for multiple HDDs (like the Orico Honeycomb or similar multi-HDD dock), and populate it with a couple of hard drives.
My question is this: If I buy 3 HDDs now of a certain size, can the ZFS pool then be extended with more HDDs of different sizes? Do I have to migrate my complete ZFS pool later if I replace all the drives with drives of a different size?
You may find this article from Klara systems helpful. I personally found I was able to use ZFS with a great deal more confidence after reading zfs mastery.
Each pool can be comprised of any number of vdevs; their topology and size donāt have to match. Each vdev is responsible for its own redundancy, and each disk in the vdev must store the same amount of data. Note that this means that you can have mixed size drives in a vdev but you can only use the capacity of the smallest drive. In my case I have a mirror of two drives: one with 3TB and the other with 4TB, so the vdev is 3TB in size.
To answer your question, say you build a pool with a raidz1 vdev from your drives. You can add a vdev of mirrored drives (two) later to increase the storage space of the pool. You could add another raidz1 vdev with larger (or smaller) disk sizes.
What you canāt do is convert the existing vdev type (say raidz1 with your three drives) without temporarily migrating the pool to another system. You can do an in-place upgrade of the disks by replacing them with larger capacity drives, although you have to do it one at a time which can be lengthy.
With 4TB SATA SSDs now reaching below $150 and starting to touch $100 on promo deals, I can no longer in good conscience recommend HDDs unless you need more than 20 TB of storage. It just isnāt worth the hassle anymore.
While yes, you can get an 8 TB HDD for $110 and a 16 TB HDD for $260, these will be so atrociously slooooooow by now it will feel like being on a race track with a two cylinder engine. Consumer level SSD NAS is now a thing, still mostly for enthusiasts, but it is becoming rapidly affordable for high end and midrange buyers, too.
I wouldnāt say I āneedā 20TB of storage. To be honest, I donāt really āneedā the 4TB I have now. But I want it. Iām content with my data running a little slower, this little bastard can encode 4K video just fine even from spinning rust, so I guess the price per TB is a bigger hurdle for me.
That is, in my mind, a good enough reason to stick with spinning rust at this point in time. I may be very content with having a ton of storage, because even though I only have 2TB used on the HDD, I am wary that this space may run out very soon.
Thank you for the book recommendation! Iāll read that (and the article too, of course).
So basically I can add other vdevs later, and point the ZFS pool to āalso use those ones as wellā? Thatās what I would love to be able to do, buy 1 drive now, another drive next month, etc. to progressively expand the storage space.
If you want to be able to āeasilyā add drives with zfs, you should do what @Zedicus advises and use mirror vdevs. That way you can add drives 2 at a time, which will not only increase your capacity, but your (theoretical) performance as well.
So if I understand correctly, I get a DAS (anything I can fit drives in), and then put in 2 drives, mirror them, and later I can add 2 more drives to mirror that first 2-drive mirror?
That way, I give up 50% of the storage for a 2-drive fail tolerance, if I understand correctly?
Okay, for everyone involved, Iāll have to pop this one feeling in and what I found that may have changed my mind (in addition to literally everyone here):
This is the one thing I was worried about. I am a greedy, penny-pinching person when it comes to storage. Drives are not cheap, and I am now in a position where I have 1 4TB drive. I donāt want to lose it, but I also have no way to back that thing up. If it goes, itās off to the data recovery center and they will charge much more.
So I believe the next plan is as follows (please correct me if I write anything stupid):
Get a rack to put drives in (4-bay, maybe, if I can find a 6-bay Iāll save up for it)
Buy 2 4TB HDDs (I believe thatās what I got in there now)
Set up a mirror vdev
Move data from my current 4TB HDD to the VDEV and keep it as a cold backup/expansion/replacement drive
This way, I get what I had before, but with plenty more resiliency and a backup!
After that, I can buy HDDs in pairs, as apparently the size difference between VDEV mirrors shouldnāt be bad for the overall storage (the 6TB would mirror each other and the 4TB would mirror each other). Now just to figure out if I can use the internal SATA drive on the Optiplex to hook up to a 5.25" HDD bay (finding external HDD bays is pain)
Am I thinking right, or did I miss some huge thing that will mess my day up?
I use the Silverstone DS380, it sits on a bookshelf in my living room.
I would think of it as 1 drive of resiliency. Maybe 1.5 drives? You can lose your pool with two drive failures (if they are in the same vdev), so itās definitely less than 2.
My initial setup was a cheap SOC motherboard and throw away case. Definitely good for learning. As I transitioned into keeping important data, I found it was worth buying more reliable stuff (IPMI, ECC, hot swap bays), but thatās hard to justify when itās your first nas.
if you do not already have stuff, i recomend an H8SGL and an opteron 6366HE. (if you find a good deal on a board with different CPU included, it is probably fine. the 6366HE is the lowest power that gen, though) this will support ECC REG RAM, is DIRT cheap, and is reasonably low power considering its age.
Toss any PCIE HBA you can find on it and you have a brilliant reliable nas. (I am still using one of these as a NAS right now)
If the data is really important and canāt be found again (family photos and the like). You really need to look into a backup plan. Raid is for uptime, not backup. I personally use Backblaze for about $7 a month and have been happy. There are a few other providers in this space.
āAmazon Glacier Deep Archiveā is another option, seems very cheap storage from what Iāve seen if you donāt plan to access the data much or at all. It is not as user friendly, but once your backup system is setup it wouldnāt be an issue.
Since you are now building a NAS proper, here is what I would get today:
Old school - 5 bay QNAP or Synology:
$649, Power draw ~60W, and you need to buy drives for this. Compare then with the Asustor Flashstor:
$449, Power draw ~30W, and six drives instead of five. Also, whisper quiet.
Now drives: What would I go for? Looking at HDDs 8TB seems to be the sweet spot between price per TB and actual price, so I am settling for the Seagate Barracuda Compute for $110 WD Blue for $125:
As for SSDs, any m.2 gen 3 drive will outperform the HDDs, but lets go with price per GB again. Crucial P3 4TB are $190 right now so let us go with that:
So, for the added cost of $65 per drive and 24 TB of raw storage instead of 40 TB (20 vs 32 if RAID5, 16 vs 24 if RAID6), you get incredible performance. Total costs, all decked out on both platforms, is $1275 vs $1590, or roughly 25% extra for superior speed, reliability and power consumption.
Furthermore HDDs are now getting more expensive to produce due to lower batch numbers. While 8TB and 32TB SSD costs will creep down eventually due to economics of scale, HDDs will inevitably increase in price, going forward.
This is why I no longer recommend HDDs for small NAS storage needs. Of course, if you require 40-50 TB of storage or more, which is rare for home use but it happens, then HDDs still make sense, but once 16TB SSDs becomes affordableā¦
Right you are. So what is the cheapest 8TB consumer drive that isnāt SMR then? I will upgrade to a $125 WD Blue but if you can find cheaper I will listen. This also means there is now a +$75 difference in price, which slightly helps my argument here.
P3 plus is nice but here we are talking storage / price not (excessive) perf / price. The only difference is the speed, both can saturate 10GbE easily.
