I am switching away from using my HTPC as a file server and am looking to leave it as an HTPC and building another PC to be be a File Server. I was thinking about using FreeNAS and ZFS. It may also host a VM or torrent client but this is not a primary concern.
Right now my current storage is 4x 8TB drives in RAID5 giving me ~24TB of storage. The plan was to build the new system and buy 4 more 8TB drives and get it setup. Transfer all the data from the one computer to the other, then pop the other 4 drives into the system and expand its storage to all 8 drives.
What im wondering is what is suitable hardware for this. Something low power is probably preferred. Is it possible to use something like one of the Ryzen APU’s for this task? Would 32GB of RAM be enough. I know I have heard 1GB of RAM per 1TB of storage but I dont know how much wiggle room there is.
For the 8 drives I know I would need a expansion card of some sort and that RAID cards are not preferred. I already have 2 of the red ASUS Aquantia 10G NIC’s available to link directly to other systems (one to the HTPC and one to my workstation).
Lastly I know SSD caching is an option would NVMe be best for this? I ask since I could have something small like a 128GB otherwise I have a number of 256GB SATA drives kicking around and could chuck a number of them in there on the chipset’s SATA controller.
Would it be wise to have some sort of redundancy for the boot drive?
Just wanted to say thanks in advance for the help. I know its a bombardment of questions. Its been a while since I have looked into a separate system for my server since I tried to only have one system on 24-7. I should be able to have the HTPC off when I dont need it now due to not needing to record TV.
If there is more info I can provide that you think would be helpful let me know.
Oh for reference the HTPC is a Sabertooth X79, i7-4820K, 32GB DDR3 (could upgrade to 64GB), GTX1060 FTW2 ICX
nothing against rackmount really. Only reason may be noise. Also for reference I am using the Seagate Archive 8TB drives so it would need compatibility for those. I know some older hardware lacks larger drive compatibility.
Also if im reading correctly are those R510’s running X58 equivalent hardware?
If we went desktop parts I should have everything but the MB, CPU, and Expansion card for the drives.
No I have no need to transcode at least at the moment and if I did it wouldnt be realtime. All of my stuff is already in my preferred format for that content, either already encoded or just straight raw.
If anything I may need to run a windows VM on there (mostly for backblaze) and then may want to run something like rutorrent just for convenience and to get it off my main system. Most of this systems job will just be to house my data and get it to my other systems.
I do not generally edit off of it currently I usually dump original for backup then another completed/edited copy when done. It also maintains syncs via resiliio sync for commonly used files across my devices instead of stuff like cloud storage and backing up my phone to the server.
As for some of the other parts of your post, yes it is advisible to use redundant boot drives for a FreeNAS setup. A mirror of 2 x 16gb or 32gb thumbdrives is both easy to setup and more than enough for your use-case. I like to use the little Sandisk Ultrafit drives that barely stick out of the USB ports. The boot devices aren’t really stressed whereas the entire OS is loaded into memory upon boot and the drives don’t sustain many writes or reads otherwise.
There is a little wiggle room with the memory requirements. While 1 GB ram to 1 TB is optimal you can get away with less. Just keep in mind that if you ever plan to expand your pool, or if you are running a VM or multiple VMs you may want more resources. I’d personally go with 64GB ECC if you want a platform that supports it. Once again, ECC is a strong suggestion but not absolutely necessary.
SSD caching or ARC and ZIL drives are not necessary in your use case. They add complexity and will not generally net you much (if any) in the way of performance gains. Other OS platforms like Unraid really benefit from cache drives, however, FreeNAS is different.
As far as expansion cards go, you are right, Raid cards are not a good option in FreeNAS whereas the system wants direct access to disks and not proxy access through another layer. In this case you’ll want an HBA or Host Bus Adapter card to handle the drives. I’ve been using LSI 9211-8i cards for years and have yet to have a problem. I’m not completely sure about saturating a 10GB connection with that card running your drives, whereas, I do not have any 10GB equipment…Yet… But you should be able to get much faster than standard gigabit speeds.
8 Total drives lends itself pretty well to a RaidZ2 configuration on FreeNAS. This is essentially 2 parity drives and all the built-in ZFS goodies. You do lose some performance and obviously storage capacity with RaidZ2 but you have to make that call depending on how important your files are to you. You mentioned Backblaze so I imagine you mean to remote backup your pool (which is a good idea). Meaning if you’re comfortable with that being part of your redundancy, you may be able to get away with RaidZ1 and regain some of that local storage space. This one’s up to you.
Looking forward to hearing back from you on your budget and any follow-up information you provide. Eitherway, Good Luck!
My budget it’s basically whatever is reasonably required to do this. As I don’t want to spend more on the most optimized perfect system as I more than likely won’t take advantage of. Nor do I was the thing that will just barely get me by.
It’s basically what would you buy for yourself if you have my needs? Based on what I’m using it for. Requirements for this especially for my use don’t seem high so threadripper and X299 should be out. I mean I guess I could use the existing HTPC but then I’d have to find it a replacement
As for the boot drives I’ve got two classic Intel x25-V40’s that would be fun to use.
I would like to try as beast I can to saturate 10G withe my 8 drives. Or as much of it as I can.
Initially I will need a RAID Z1 for initial array size. Could thing bec changed to Z2 without affecting data
If you mean to re-shape a z1 pool into a z2 pool, that means you need to migrate everything off it, then remake the entire pool and remigrate everything back. So without affecting data, no.
Just plan ahead and make the pool the way you are going to have it in the first place.
Good to know. I’ll have to go Z1 then since I’ll only have 4 s drives starting out and will need enough space to pull the data from the other 4 drives that are in RAID5
For now I’ll be sticking to 32GB of RAM since I have that kicking around. I may go to 64GB and even ECC once I have a bit more money to spend.
If I recall correctly Ryzen supports ECC so that would allow me to do that since it doesn’t require it persay. Do the APU chips as well? I ask since that would get me a GPU without needing to take up a slot but does that then also limit my available PCI-E lanes? I forget. Would one of those even be powerful enough as a GPU.
Also me wanting to use those old Intel SSD’s is more for throwback/nostalgia but if there isn’t and issue I’ll make it happen
Would it be wise to different drives that the software I mentioned above runs on?
Lastly what would an 8 port HBA that would be able to saturate a 10G connection be?
The APU chips and ECC wont work, technically they are capable but no motherboard supports them unless they are a Ryzen Pro chip and those are made of unobtainium. I too had the exact same idea as you did, being able to not have a dedicated GPU when I don’t really need it, but be aware a ryzen 2200g for ex. only has 12 PCIe lanes, where as a Ryzen 3 1200 has 20, so you would just save the price of a GPU and space in the case.
32gb of memory is overkill for such a small build. I currently run a NAS that has 8x8TB and has 8gb of ECC ram and it works fine at least in 1gb speeds, the thing is that you don’t want to run deduplication when you have low quantities of ram and in my own case I’m going to upgrade it to 32gb when I have time and money.
The 10gb networking is a whole another beast, if you arent running point to point 3 meter max connections, expect to pay $50 and up for each link module that you need 2 for each connection and if you need actual 10gb network, you need CAT6A if using copper cables or fiber optics. Mikrotik did recently release a reasonable priced 4 SFP+ port switch / router (CRS305-1G-4S+IN) but unless you get that you are looking at $500+ for the base switch chassis with no sfp+ modules.
So Ryzen will work for this then? Probably go non APU die to the lack of PCI-E on the APU. Is there a recommend board. I don’t need anything special. Are all Ryzen boards ECC compatible? I was thinking probably a base model ASUS B350 or B450 since they are some of the most stable generally.
As for the 32GB I have 2 set of 4x8GB kicking around so I don’t need to buy it. I would have to buy ECC memory so. I’ll probably hold off until the upgrade to 64GB
I already have 2x ASUS XG-C100C one will be direct wired to my Workstation the other direct wired to my HTPC. This setup is currently working between my HTPC/Server and Workstation via a CAT7 cable.
I guess what I was more worried about is a HBA that can handle my drives to get the most speed out of them is that makes sense.
You can install on a one of the non-apu’s, have the GPU in while you install the OS and set the webgui’s IP address. once thats done, you can swap out the GPU for an HBA or whatever. This would allow max bandwidth from the HBA and the 10g NIC. Trying to remember if you need X470/X370 to have both of the X8/x8 slots connected to the CPU, or if B450 is sufficient. I’d check when looking at motherboards if there are at least 2 PCIe slots that are connected directly the CPU
Additionally, the 1gb per 1tb of hard drive space is for deduplication. Don’t bother with dedupe, it’s not worth it for HTPC usage. Additional RAM does mean larger ARC though, so its not like its going to be wasted at all.
All ryzen CPUs have ECC capability but whether its implemented for them depends on the motherboard manufacturers (which is why ecc wont work with an APU) and not all motherboards are created the same, so search online before you get a board to make sure someone has already tested that the ECC works.
Truthfully I would. Like to find a board that is X8/X4/X4 since I have 2 10G NIC’s (x4 each) and would have the one HBA (likely x8)
As for HTPC this won’t be the HTPC I will use the existing PC for that. This will be my file server. Maybe run a VM and some of the software I mentioned.
What is deduplication and why would it be important? I’m at work now but I can do research when I’m done, otherwise if you have a link on the info that would be great.
Will do. I would also like to try and fine a board that will do X8/x4/x4 off the CPU which could be slim picking. If anyone know of one like that for Ryzen please share. I know some of the latest Intel stuff is doing that.
Deduplication is exactly what it sounds like. Essentially, it is a background process that constantly searches your array for duplicate files. In the event it finds duplicates, (correct me if I’m wrong) it creates a symbolic link for that file in the locations the duplicate is found and only keeps a single version of the file on the NAS itself. It aims to keep large arrays full of duplicate files more cut down on storage. Likely not something you’ll need and yes it’s very memory (ram) intensive.
I believe that iXsystems actually recommends 5GB ram per 1TB storage for dedupe.
What would be an HBA that could take full advantage of those 8 drives for reference? I should have PCI-E 3.0 x8 lanes available.
Got a response on Twitter from Wendell and he said The Crosshair 7 Hero can have a x8/x4/x4 from the CPU. Its no explicitly stated in the manual but he knows his stuff so ill give it a try. To clarify this was split out into a M.2 so i would need an adapter it is not the lower slot.
Looks like your best bet for an 8 Drive NAS for compatibility, speed and price is going to be an LSI 9300 - 8i. It’s a PCIe 3.0 8x HBA thats SAS3 compatible and can control 8 drives. You’ll also need 2 x SFF-8643 to SATA breakout cables to wire up the drives to the card. They aren’t cheap per-say (like $200ish on amazon) but you may be able to find a deal on ebay for one.
Another thing to remember is future expansion. If you ever intend to throw a second vdev of another 8 drives into the mix, you’ll want to go with it’s bigger brother the LSI 9305 - 16i. This is the same spec but with 4 SFF-8643 connectors capable of running 16 drives. But these come at a premium of about double the cost or maybe even a little higher.
not always… you can just get a case that has a backplane built into it… or you can get a sas expander card separately and use with the HBA you had/have.
