I’ve been watching my current FreeNAS pool sit at around 75% used space for the past while. I’m not going to hit the end of it any time soon, but it has made me start digging into how I can expand down the road.
Looking at my current setup (pcpartpicker list) , I’m not exactly happy with my choice in cases, and adding more sata ports would be difficult. The 2 gpus and usb expansion card required for my virtualized Windows instance don’t leave any real room for a SAS card. And my thoughts of adding hard drive mounts where you’d normally put water cooling stuff in my Corsair 1000D don’t seem very workable.
So I’ve created a pcpartpicker list based on the idea of moving my NAS instance to it’s own machine. The list is here
My main use is storage for my Emby server, storage for my game library, and just general storage for anything that doesn’t need SSD speeds. Long term I have thoughts of trading space for backups with other home NAS owners. I also just have fun building and running things like this.
Here’s a list of questions I have about my parts list:
What kind of processing power do I really need for FreeNAS? I know the FreeNAS forums end up recommending Intel a lot, but I’m a fan of the upgrade potential in the AM4 socket on AMD’s side, so that’s why I chose a 2200G. (I also wonder about using Proxmox to virtualize things and then running my firewall and ad-blocker on it…)
As far as I can tell, B450 seems to be the sweet spot for motherboards, not expensive, but not cheap. Would FreeNAS have any issues there? What about virtualization support? Things like splitting out IOMMU channels so I can pass through pcie devices and HDDs.
I’d install the OS on a small M.2 ssd. No real reason other than price for the Kingston A400.
Are NAS drives actually worth it for a home NAS?
Finding a case with room for lots of hard drives is kinda hard nowadays… The Rosewill THOR v2 seems good, but I’m open to alternatives. I do like that I’d be able to fit 2 SilverStone HDD enclosures in it. (I’m currently in a small 2 bedroom apartment, so going with a server chassis would be too noisy. )
Would the LSI LOGIC SAS 9207-8i Storage Controller LSI00301 cards work? I have no experience with SAS, but LSI is a brand I kept finding when doing research on what FreeNAS works with.
Would the SilverStone cables be good?
Does anyone have any experience with the SilverStone hard drive enclosures? I like the idea of hot swapping in the event of hdd failure, but would I be better off finding a way to do this without enclosures?
Your parts lists look fine although the case may not give a lot of airflow with the two disk cages running at full tilt.
On your specific points:
CPU and board - for NAS only (not a converged VM host) you don’t need vast power and your scrubs will run overnight so who cares if it takes a couple of hours. If you are running VMs then you may want to consider a CPU / platform that is geared for virtualization, and actually an OS that is too (freenas doesn’t do pass through). The 2200G is a low end consumer part and whilst AMD support virtualization all the way, all of the enterprise tooling assumes you will have a Xeon. iOMMU support is ok on b450 mobos but check reviews of the specific one you are going to buy to make sure it has good groupings.
Drives. Yes to the small SSD for host. It will be a waste of space on 120gb but as they are cheaper than water these days it is better than a flaky usb controller. Just backup your config. For storage drives I use a mixture with (WD) reds, greens, blues and even a weird white one in my Nas. It’s been running fine for years with just one (Green) disk dying. I reflashed the greens so they don’t sleep though. Note I use 5400 drives because I like to sleep at night. I’ve not had an iron wolf drive ever so cannot comment on reliability but expect some rage comments about Seagate from someone in any freeNAS thread.
case and cages - per the opening comment running two disk cages makes the case 1) warm and 2) heavy. The fans on those cages tend to be loud as well. It will work and give you limited features but if you are going to target 10 drives you should be working on the assumption that one will break within 3 years, so having proper enterprise backplanes may be better use of your sas controllers. I’m not sure I agree with your point that server cases are louder than consumer towers… Just swap the fans. If space is really that much of a premium is a 10 disk NAS the right solution Vs something smaller. There are some neat 8bay cases with hotswap backplanes available. I’ll edit this comment when I remember the name.
sas controller. Don’t know for sure if that controller has any quirks as I don’t use it but generally speaking LSI controllers work fine with freenas and as long as the breakout cables have the right connector, it is all just plug and play. My only comment on controllers is new ones tend to be vastly overpriced, and used ones are stupid cheap. If you are smart with your eBay searches you can save enough for an extra hard drive or more ram.
sundries. You haven’t mentioned RAM but if you are looking at Ryzen and the bios is fairly recent then ddr4 3200 is a good price these days and will give you capacity. For that size array you really want 16gb or 32gb if you can afford it. If you don’t mind compromising on old/ used hardware an ivy bridge era Xeon with 64gb ECC will be cheaper than buying 32gb ddr4. Depends if you want warranty.
You can’t hook up 2 SAS controllers to a 2200G because it only has 8 lanes of PCIe and, if I’m not mistaken, not many B450 motherboards (if any at all) can split the PCIe lanes coming from the CPU.
So going for an Intel system would really make sense for you since you could get a cheap CPU that it’s not skipping on PCIe lanes and can run two x8 devices.
If you want to go with Ryzen you have to buy at least an X470 motherboard and an R5 1400 (you shouldn’t go with that CPU, very weak first gen Ryzen, not worth. It’s just an example).
I would rather go for a r5 1600 or so instead of the 2200g.
The only reason fro the 2200g I can see would be the GPU, which I have no clue about if it is even supported or useful in this hole scenario.
I don’t think so, which is why I’d switch to a different one.
The SAS controller should not be in raid mode for freenas.
The Pass through mode on those is called IT mode, and you can flash them to that usually.
Beside ebay, have a look at the STH forum for used stuff, you will probably find what you are searching for.
the ryzen supports ECC ram and ram speed should be irrelevant for this task so why not get this.
first gen zen is not bad in general is has some serious short comings in the form of AVX2 else is like 10-20% IPC that not much taking the price into account.
for ryzen APUs with ECC you need the pro GPU. some AM4 boards are supposed to run fine in headless mode so no need for an APU.
another recommendation for the disk get eternal wd disk asnd shuck them these pretty much always have WD white in it these are pretty much the same as WD RED the nas series so they are a great bargin.
depending on where you life the guarantee is untouched
so i why SAS in the first place?
SAS expander is another option, instead of adding an additional HBA (they’re often well cheaper too).
On the subject of HBAs, I’ve found it rather a lot less painful if you can just get a card that is already just a HBA (so no RAID) so you don’t have to go through the flashing dance (which is in many cases only needed because of FreeNAS/FreeBSD driver limitations, not because the card doesn’t do actual JBOD, eg. Dell PERC310).
As for cases: I have to agree with Airstripone, SuperMicro, for example, has “SQ” (SuperQuiet) versions of much of its components, including cases. The SuperMicro server in my office makes less noise than my desktop (which is in a quiet Antec case). That said “cheap” isn’t what I’d call the chassis (mine also “only” holds 8 hot-swappable drives).
“Regular” server chassis can also be made quieter if you replace the fans (and PSU, usually), this requires a big enough chassis though, trying to make a 1U chassis with a Xeon furnace in it “whisper quiet” is an exercise in futility. But 3U and 4U chassis can definitely be made quiet as there’s lots of room in them so larger, and quieter, fans can be used. (I imagine 2U can also be done, though options for CPU coolers might be quite a bit more limited, haven’t really done much research on that front though)
Thanks to everyone who has responded so far. Several things were brought up that I had overlooked.
I’ll do some more research on how many pcie lanes different chips and boards support, and how that will effect storage.
At work I’m a Linux sysadmin, but I started right when we were moving to VM’s, and I’m not managing the hardware for our vsphere, so my hardware experience is slim. Our data center is plenty noisy, and every server I’ve seen on YouTube has been noisy. So that’s why I assumed server chassis are noisy. But it sounds like I should check my assumptions and do more research.
I read a bit about needing to flash firmware on SAS controllers. I haven’t dug into it very much yet, any tips on how to tell if the hardware I’m looking at would need flashing?
@marelooke Is a SAS expander a different kind of pcie card than what’s on my list? Or are you talking about a server kind of thing that lets you add more drives to your SAN? Also, could you define HBA for me?
@Huhn_Morehuhn I had run across shucking in a YouTube video before, but the guy had to do some kind of pin bypass, I think it was called, to make them work. You know of any good resources on the subject to help make sure someone new doesn’t mess up?
No - it’s the other end of a cable from the pcie card. Think of a power strip for your mains electricity. One plug goes into your wall, 4 or more plugs come out. Your HBA will have one minisas cable coming out that would normally support 4 drives. A SAS expander will multiply that to 16 or more drives. Some do clever things like aggregate data or raid. I would say they are not as simple to use for novices than just adding a second HBA.
Usually only needed for very large drives but you literally tape over a pin to force it low. This disables power saving mode if you are using a SATA controller. Plenty of videos on YouTube about it. What none of them mention is that if you just use a SAS HBA like the one you were looking at then it is irrelevant, as the ‘feature’ is a SAS sleep mode and the controller just deals with it.
No it will be fine. The LSI SAS 9207-8i Is just a HBA, not a raid controller. Per my note above though you will pay $100 plus for that card. A used one on eBay etc will be a third of that and just as good.
Wasn’t aware of expanders doing fancy things, most of them it’s just a matter of plugging a HBA into the expander and it sort of just functions like a switch in that to the system there’s no difference between things directly attached to a HBA or to an expander. The HBA still manages all the disks.
The nice thing about some of the PCIe expanders though, is that they can have an external power plug (as in, they don’t have to rely on PCIe slot for power, which is the only thing they use it for anyway), meaning you could mount them elsewhere in your system and free up a slot.
I recently built a dedicated FreeNAS server. I used an Antec P101 Silent case which I’m really happy with.
It has 8 internal 3.5" drive-bays. [Tool-less, but not backplane or hot-swap].
Detailed component list below.
I feel/know I kind of overspec’d it, but I keep telling myself it won’t need upgrading for quite a while… (other than adding a couple more HDDs).
I run it as a single ZFS pool, comprised of a stripe of three vdevs [each vdev is a mirrored pair].
Antec P101 Silent Case
ASRock Rack X470D4U
Ryzen 7 1700
Noctua NH-D15S CPU cooler
32 GiB ECC RAM : 2x Kingston KSM26ED8/16ME 16GB DDR4 2666
LSI SAS 9211-8i 8-port 6Gb/s (IT-MODE)
10Gtek X520-10G-1S-X8 - 10Gb SFP+ NIC
Seasonic Prime Gold 550W ATX PSU
6x WD Red (White Label) 8TiB drives
Switch: MikroTik CRS309-1G-8S+IN
Ah, got it. So the LSI SAS 9207-8i is an HBA. A SAS expander would let me add more than 8 drives with just one LSI SAS 9207-8i. From Broadcom’s page about that card, I could go all the way up to 256 drives…
I read that a while back, good to read it again though.
Hmm… Now that I understand SAS a bit better, I’m starting to wonder if I can fit a SAS card with an external port into my current setup, then use an external SAS enclosure like the Silverstone TS431S to add drives.
My problem with my current setup is my gpu’s get in the way of any new cards, even though I have empty pciex16 slots. I think I might be able to get around that with a pcie extension cable.
Anyone know of an external SAS enclosure that has room for multiples of 5 drives? I’m currently using RAIDZ1 if I remember correctly. So I’d like to add drives 5 at a time.
Yep. Any disk shelf that is 2U will hold at least 12 drives (3.5"). Scales linearly from there. Search for disk shelves on eBay etc. Most come with a built in expander or at least power for one.
Of course you are now getting into rack territory which adds more points of failure, maintenance, noise, and low WAF. But if the rack goes in a ventilated closet, basement etc, along with your network gear and UPS, then you may find you have great freedoms with your 1000D case and consider something smaller and quieter for the desk…
there is a power story.
yes you may need a molex to sata loose all your data cable. there are guides about removing a pin or taping it… just no…
a decent molex to sata and you are fine. the HDD stops working if the 3.3 rail has power and molex doesn’t have 3.3 rail. there is no question if this works and it’s 100% spec confirm.
not sure if newer PSU can handle this or how this exactly works commonsense said to me no.
i still have a problem understanding why you want sas and why you are willing to spend money on as PCIe addon cards for 5 drives and why the PCIe lanes play such a rule the hdd will never reach 12 gbits not even close to it.
even PCIe 1.0 x1 can do 250 mb’s you don’t need all lanes to get 100 % out of your HDD. 4 disk with 1 gigabyte/s and you are totally fine which is about 1 lanes of PCIe 3.0