Looking for advice, building NAS/Server

Hardware Hub Build a PC
, ,
1

I’m building a “real” NAS, looking for some advise and critique. I’m already using my PC as a NAS/Server/PC, but now that I upgraded my PC I could use the old stuff and buy somethings to create a real NAS/Server. What I have is a TrueNAS scale with 2 VM; one vm for my Personal Computer (Linux/vfio/passthru-videocard) and one headless vm I called Server. “Server” is a Linux vm with dockers, a docker with nextcloud and a docker with a Steam-cache/Games-nfs. All that is runing on a 8TB pool rust spins mirrors for data/replication and 2TB nvme pool stripe for the VMs Zvols.

I’m building a new NAS/Server, so I can pass the “Server” vm and data to it. Will be running TrueNAS scale and I want to add a Plex docker to it. I already have have many old part that I want to keep on use and just buy the necessary parts. I want to focus on good Read speed, I’m ok with write speed and storage capacity compromise for more read speed and redundancy.

I already have motherboard, CPU, memory, boot drive, case and psu. What I need is cpu cooler, Hard Drives, SSDs, IO connectivity and a 10Gbit Land card.

I’ll be bringing the following with me from my old system:

  • Ryzen R9 5950X
  • ASRock B550 Taichi
  • 128GB Ram - Patriot Viper Steel DDR4 RAM 128GB (4X32GB) 3600MHz
  • 128GB boot SSD
  • 2x 500GB 2.5 SSD for VMs
  • Cooler Master HAF XB EVO - DIY Modified to fit 2x Triple 5.25" Bay, beside the 2x single 5.25 bays and the 2x 3.5" hot swap
  • A SilverStone Technology FS305-E Three 5.25" Device Bay to Five 3.5"
  • A Icy Dock ExpressCage MB326SP-B 6X 2.5-Inch SATA SSD in 5.25-Inch Bay
  • 1000W PSU
  • VisionTek Radeon HD 5450 1GB (Optional, I will use the cli)

I want the NAS/Server to be my Next cloud, Steam Cache and Plex. That way I can use my PC to watch something from the plex while updating games, while my steam deck download a game from cache and at the same time family members could watch plex collection from smartphones and TVs around the house. What I think I gonna need to buy is:

Type Item Price
Network 10Gtek PCI-E NIC Network Card, SFP+ Port, with Intel 82599EN ~$34.00 @ Amazon
CPU Cooler Noctua NH-U12S SE-AM4 ~$75.00 @ Amazon
12x Hard Drives 12x Seagate IronWolf 12TB HDD ~$200.00 ea =~$2,400 @ Amazon
12x SSDs 12x Crucial MX500 500GB 3D NAND SATA 2.5 ~$50.00 ea =~$600 @ Amazon
HDD Bays SilverStone Technology FS305-E Three 5.25" Bay to Five 3.5" ~$146.00 @ Amazon
SDD Bays Icy Dock ExpressCage MB326SP-B, 6X 2.5-Inch SATA SSD ~$82.00 @ Amazon
M.2 Expasion Card ASUS Hyper M.2 X16 PCIe 4.0 X4 Expansion Card Supports 4 M.2 ~$70.00 @ Amazon
4x M.2 to 5 Sata 4x SilverStone Technology ECS07 5-Port SATA Gen3 6Gbps ~$60.00 ea =~$240 @ Amazon
Total ~$3,647.00 - w/tax ~$4,066.41

My Idea is to transfer the “Server” vm and make a pool for data and replication the same way I have on to the new NAS, but adding a plex. If I rip all my blue rays I may need 25TB, games is like 3TB, next cloud is less than 1TB and 1TB of replication. I guess I need ~30TB to start and some space to grow. I was thinking on 6x 2-way mirrors 12TB HDDs of Data (256k block) and 6x 2-way mirrors 500GB of Metadata (up to 128K). That should give me ~66TB storage, a bit more than double of what I need. I may think on doing a 3-way for ~44TB, but I dont know if is worth it, to only have 14TB of free space. Any advice?, Is my plan ok?, Am I doing something wrong on this?. Oh, my Budget cant exceed $4,200.

2

According to the manual of your motherboard this wouldn’t work as you lack PCIe lines and bifurcation support. I would also highly recommend that you consolidate the amount of drives since it realistically wont be a bottleneck anyway.

NIC: Ancient, consider something like https://www.serversupply.com/NETWORKING/NETWORK%20ADAPTER/2%20PORT/BROADCOM/P225P_311355.htm instead ?

CPU Cooler: Scythe Fuma 3 gives you better value for example and is 25$ cheaper

HDDs: Consolidate these, you’ll save a lot of headache with less drives not to forget power and thermals. The IronWolf series are a bit crap, if you’re deadset on Seagate Exos series is what you’d want to look at. 8x - PROVANTAGE: Toshiba MG09ACA16TE Toshiba Hard Drive MG09ACA16TE 16TB 7200 RPM SATA 6GBPS 3.5 512E Bare ~2280 USD
I guess you could go back and forth about but given the available hardware and the data you’re going to store RAID-Z(1) is probably “good enough” and will save you a lot of headache.

SSDs: Not sure why you’re going for tiny ones, I’ll just make everything a headache.
SATA is also “dying” so it’s not really as cost effective as before however PCIe switches are expensive and not worth the hassle in this case.
6x https://www.amazon.com/SanDisk-Ultra-NAND-Internal-SDSSDH3-2T00-G26/ ~780$

Controllers:
The motherboard have 8 SATA ports, 4 from the SB and 4 using ASM1061 (these will be “slow”).
You can either get a LSI HBA, if you’re not going for the ones from .cn (I would be very cautious) it’ll set you back ~50-200$ for a refurbished one (SAS2008 (these are EoL and should be avoided) or SAS3008) and I’m not sure how well these handle SSDs (TRIM) etc. Another option would be getting ASM1166 which is probably the route I’d go in this case.

The handbook specifies that you have 8x/8x/4x using the PCIe 16x slots,
So…
Slot 1+2: GFX + NIC
Slot 3: Controller Silverstonetek ECS06 or Amazon.com: GLOTRENDS SA3026 6-Port PCIe X4 SATA Card, Including SATA Cables and 1:5 SATA Splitter Power Cable, Compatible with Windows,Linux,Mac OS,NAS : Electronics and one of the M.2 slots (see Short Review: Edging ASMedia 1166 PCIe Gen3 x2 to 6 x SATA HBA Chipset. It doesn't suck 👍 )

If it would support bifurcation you could utilize the PCIe slots better but it’s undocumented…

I would honestly ditch the current case and grab a Fractal Design R5 or possibly XL and skip the hotswap bays because of cost efficiency.

3

I was going for that Nic because I know the drivers are available. But the Broadcom P225P sounds better, my router have 2x SFP+ 10Gb/s ports but one day I would upgrade.

For sure I’m switching to Scythe Fuma 3, looks cooler, pun intended. Plus have good reviews. Thanks

You are right that it does not have straight bifurcation, but I found a Raid option to do 4x4x4x4 on the bios. Like its says on this forum: B550 Taichi and x4x4x4x4 PCIe Bifucation - ASRock Forums . I thought I could use a the Hyper, but now that I think it will disable my second x8 that I need for the LAN card, plus could force me into some raid options from the board.

I still think that I can do it if I dish the GFX, I can run it headless. I can

Slot 1: Nic and Slot 2: GLOTRENDS SA3026 6-Port
Slot 3: another GLOTRENDS SA3026 6-Port.

I have two M.2 onboard where I can put M.2 to SATA 3.0 Adapter 6-Port, with Chipset ASM1166 on each one.

That should give me 24 Port plus 4 from the SB. More than enough.

My Cooler Master HAF XB EVO have 2 floors and is super roomy. I can access the HDDs bays on the second floor from the top, because I modified it to eliminate the frontal fans and put the bays there. and I can access the first floor from the side where the SSDs bays will be. Because there is not Video Card the power and thermals should not be a problem, by my calculation is 170W if all HDD and SDD are running at maximum. The Bays are mostly for comfort, only 1 Sata Power cable per Bay, so way less cables, I can label the data port on the back of the bay to match the cables and they come with fans, plus I have output fans on the case.

I still think that 12 disk for data and 12 for metadata is the way, because I used to have Raid-Z(1) on my PC. Three Disk, while the sequential reads where at 300MB/s the random dropped to 30-50MB/s. I added a disk and converted it to a 2x 2-way mirror instead of Raid-Z and my random reads when to 90-110MB/s. After that I added 2x 2-way SSD for metadata and my random read don’t drop from 200MB/s. I choose the 500GB disks because I’m using the recommended 3% of of the Data as metadata. And it work well for me on my setup, should work well on the new NAS. More Disks mean more Read speed and redundancy. But I may drop to 10&10 disk, instead of 12&12. because of diminish returns on redundancy on 2-way mirrors.

Thank you very much for your advice and your time :-). It helped me alot to change many things that I was doing wrong. It gave me more to think about before committing to the cart, I still have another week before I click buy.

4

Have you verified that it boots without a video card? Because that isn’t necessarily the case and also given the nature of not having any kind of debugging interface at all (serial) it would be very annoying.

About the thermals, you’re going to need quite good airflow in that cramped space, removing front (intake) fans and adding more devices generating heat will not work in your favour. You can of course try but don’t be surprised if you run into thermal issues and/or overheating.

Unless you have a rack mounted case airflow in howswap bays is bad a best and fan(s) barely make a difference unless you’re running a Delta/Sunon which will be very loud. I’ve gone this route myself before and it’s not worth it :wink:

More disks doesn’t necessarily mean more read speed as you have latency etc to take into account and your main limitation will be your network interface anyway in many cases. Given at least your stated use case it doesn’t need to push I/O like crazy at the end of the day and more devices also means a lot more of potential failures. Your numbers also sounds suspiciously low but that may boil down to controller and/or HDDs used.

I would still move away from the Ironwolf series to Exos and/or Toshiba MG-series because better MTBF ratings etc.

5

Yep, just tested right now and will not boot without a video card. Well I put the video on the Slot 3 and still have ports for 10&10. I will change to exos.

I don’t think thermal will be hard on my use case. My Next cloud only handle cloud storage for smartphones photos, backup contacts and backup of my documents. The Steam-cache is only for updates, If I need to install a game on my steam deck should be able to pull from the NAS and my PC at the same time. And I will limit the plex to do 720p unless is the living room TV that can be 4k. I will keep an eye on thermals, if they go too high, then I upgrade the case.

6

Problem solved, the ASMedia ASM2812 Chipset, It does NOT depend on PCIe bifurcation supported by MOBO. Gen3 X8 to Dual M.2 Card

Also found the correct distribution of disk to minimize the bottleneck of the ASM1166 pcie 3 x2.

SB port | 3x SSD + 1x HDD
SB M.2 | 2x SSD + 3x HDD
NB M.2 | 3x SSD + 2x HDD
PCE M.2_1 | 2x SSD + 3x HDD
PCE M.2_2 | 2x SSD + 3x HDD

The latency won’t be great, but I can live with that. I found the Toshiba MG 16TB at a great price in newegg and bought them. I decided to go 3-way mirror for best redundancy.

Thanks for the help @diizzy , you’ve improved my build tenfold.

7

Nice! :slight_smile:

Yes, however they seem to be a bit troublesome. If anything the PLX-chipset(s) seem to have better bandwidth than the ASMedia ones.

8

Hi @Nemus
Your NAS build looks solid. The Ryzen R9 5950X, 128GB RAM, and 6x 12TB HDD 2-way mirror setup provides great processing power and storage redundancy. In my view at $4,200, this is a robust, future-proof solution that balances performance, data protection, and cost.

9

Hey @Nemus
Building your Jellyfin media server with the Fractal Define 7 XL case is a solid plan. This case offers plenty of space for your 5-drive RAID 5 setup, with good airflow and flexibility for desktop components.

The case provides excellent room for expansion and will handle your media server needs well.
Consider alternatives like the Phanteks Enthoo Pro 2 if you want to compare options, but the Fractal Define 7 XL looks like a great starting point for your home media server project.

10

^ The guy about looks like a AI bot, spewing redundant or useless comments. If you are real, I apologize, but that how those comments looks to me.

3 Likes
11

Hey @RasulGreen You’re looking a bit like a bot.

If you’re real, what’s 2+2?