Hello level1s,
TLDR: I want a cheap 8 drive minimum storage solution. I might need a SAS card solution for pcie x4 slots, and will require an external SAS/SATA enclosure to expose the drives individually via a fast USB interface that can be picked up and moved regularly to cold storage.
Presently I am swapping out my Asus b650-plus for a Gigabyte X870E AORUS PRO ICE, and while I have a bunch of things shipping I figure I might as well knock out my storage solution that I wanted to upgrade. Presently I have 2 SATA HDDS running in a zfs mirror, all backed up to a single SATA HDD that I connect via a dock then move to another building for cold storage. I want to upgrade to a 8+ disk RAIDz2 based on how many disks I can find in a single external enclosure that is reasonably moved around to duplicate my stuff also formatted with zfs.
The complications come in fast when trying to get the best value per GB where someone more familiar with the market could help a lot. Expansion free on this motherboard that I’m not using for SAS compat if that’s the cheapest are the m.2 chipset slot at 4.0 x4, x16 slot at 4.0 x4, and x16 slot at 3.0 x4. Which is a problem as many SAS cards are x8 and now I need to know if x4 would be sufficient to use them power wise. Bandwidth is not a large concern as I will be caching to my 3 nvme BTRFS RAID10.
Additionally I don’t see a lot of external enclosures that provide SAS compat (or any really) or only use USB 3.0 when I could be using USB 4 now, but USB 4 is new so 3.2 is all that likely exists. If I need to make/get some weird adapter for some enterprise enterface that’s also fine, as long as it can be plugged into any PC and accessed via USB.
I’m not sold on SAS being worth it considering the lack of external enclosures that support it.
Your market knowledge here is greatly appreciated!
Ok, so if I understand you correctly, you want to replace your current AM5 system with something better and use the spare parts as a file server?
For case, if you want to go cheap, the Fractal Design Define R5 is available as one of the cheaper cases:
Now, I will assume the Asus TUF Gaming B650-Plus, but let me know if you used the Prime board instead. The only practical difference in this case is that the TUF board has a third m.2 slot, but using that disables the second x16 PCIe slot. Otherwise the two are pretty much identical.
Either way you will want an expansion card, and I do recommend a 6 port SATA m.2 adapter such as this one:
So add that to your second m.2 slot and you have ten SATA ports to contend with. That should be good enough for what you want to do. SATA is a dying standard but it is not dead yet, and HDDs still hold the price per GB crown.
However, it is highly likely that by 2030, we will see a shift to 100% flash storage everywhere. Hard drives just cannot compete with storage sizes anymore, 8-12TB is the practical limit for them. More than that is just too slow for most applications. That is not to say there aren’t niche use cases for harddrives, but SSD is overtaking them already with 64 TB drives out in the wild and 256 TB drives on the horizon.
So SSD price / TB will plummet the coming years, while HDDs have gotten about as good as it gets. Therefore, do consider all flash storage as well. You do not have to build a full NAS all at once. Just food for thought. Start with 3 drives and work your ZFS pool upwards.
I see how you got confused on that I apologize, no I’m replacing an old motherboard due to having random issues with it resetting my whole system every now and then. Since I’m forced to replace it I decided to just upgrade the I/O of my system with it. I plan to put all these drives on the gigabyte board.
I planned on archiving a lot of data I wasn’t able to fit before. As for ZFS raids, I’ve never really had any issues I’ve used a RAIDz2 on 6 3tb drives in the past.
Seeing how both of you recommended just going with SATA is it not worth the cost difference of going with SAS drives? Additionally I’m still not able to find many if any SAS enclosures that expose 8+ drives over USB so that is likely the deciding factor. I don’t want a 2nd pc always on, I’ll conjure up a mini-pc just to expose the enclosure to the local network if I take my pc someplace else for an extended period of time.
Sas might be a better option, But i can’t find x4 cards that quickly. Most of them are x8. I think if you want sas you should go with a x16 slot and one x8 slot
Unfortunately I wanted 3 nvme’s on the cpu pcie lanes leaving only the option for x4 lanes for any card on the chipset x4 from the m.2 slot, 4x from the pcie4.0 x16 slot, and x4 for the x16 pcie3.0 slot. I can run devices off all 3 simultaneously. I did some precursory searches for X4 cards and it’s hard to tell if some of the x8 cards are fine when ran at x4. If you have more intuition here I’d appreciate it.
I’m not concerned about rw speeds so much. I planned to leverage L2ARC, SLOG, and special devices on the 3 nvme btrfs raid10. Additionally the storage won’t be doing much sequential I/O anyway mostly fetching smaller individual files.
Yeah I’m pretty settled on just going SATA at this point, SAS on this platform and then finding a proper enclosure is just more than it is worth. Going SATA is very simple.
Icydock seems expensive from what I saw, I wound up going all on ebay with USB 3.2 3.5" SATA Hard Drive Tray-less Docking Station - Sabrent to store 10 external hard drives, Rosewill Hard Drive Enclosure - 3 x 5.25" to 4 x 3.5" Hot-Swa | www.rosewill.com to add 4 additional 3.5" slots to my pc case, and the cheapest pcie x4 6 port sata card to use with the existing 4 sata slots on the motherboard. I have not ordered the drives yet I’ll have to do my own research and deal hunting for that. The main complication for me was figuring out if SAS was even really reasonable for what I was wanting and it doesn’t seem like it.
Thanks all for the responses that helped me decide to just go with SATA!
Totally agree that flash is getting better, faster, and cheaper.
That said, we are still seeing capacity innovation on the hard drive side. 8-12TB is definitely not the limit. I see a lot of NAS drives in the range of 14-20TB.
In addition, hard drives continue to be far far cheaper per TB. Diskprices.com is showing 14TB Ironwolf Pros for $214. $15.29 per TB. (This is the least expensive price per TB HDD I saw this morning.)
Alternatively, when searching diskprices.com for SSDs, the cheapest per TB I see come in at about $42 per TB.
That is still a big divide in the cost. See here if someone wanted to build a 14TB ZFS pool. If using HDDs, mirrored pairs is reasonable practicable since the disk sizes are so large.
Yes, this is the case… But you are ignoring two key factors here.
HDD market is squeezed from two directions. One is capacity - HDD capacity might grow another 20%-50% until 2030. SSDs might grow another 1000% by then. HDD manufacturers are talking about 100TB drives by 2030 - SSD manufacturers are talking about 1 PB drives within the same timeframe. Once capacity overtakes, HDDs are facing a really tough and challenging market.
But the biggest reason why HDDs are smoked - transfer speed is stuck at ~200MB per second in read/write performance. This is still fine for 2TB-10TB, but at 20TB, it’s just becoming too slow to really work efficiently with the data. “Just Parity-Stripe them, duh!” - well, that’s essentially peeing in your pants to stay warm for the winter. Maybe a good short sighted move but a terrible long term one.
Imagine still working from 52x CD-ROM drives in 2024, that’s about how antiquated HDDs are about to become.
What does this mean for new HDDs? Well, new products costs a lot to bring to market. With fewer customers buying the new products, they will not decrease in price as rapidly. I am not seeing any 32TB HDDs around, are you? And SSDs have 64TB drives in the wild already.
But, all of this is going to play out during 2025-2028. Today, HDDs are old and deprecated - but not yet obsolete.