NAS Upgrade Suggestions

Hardware Hub
, ,
1

Hello! This is going to be wordy, so if you feel like reading it is appreciated.

I have been looking to upgrade my NAS machine for some time, and have been perpetually waiting for one reason or another, and I think It’s time to source community feedback

For context, here’s what I have currently:

DS415+ with 3 x 12TB seagate Ironwolf Pro in Synology Hybrid Raid

DS216J with 2 x 4TB WD Red in Raid1

I originally moved from the DS216J to the DS415+ as primary due to the networking and compute bottleneck on the DS126, and for docker support as well as more bays for better redundancy. The DS216J is now functionally a backup for critical files, and will be being moved offsite later this year.

Things I really like about Synology:

Low Power usage
Interface is easy
Small Form factor, even rack units.

Things I really hate about Synology:

Relatively Low Compute power
Proprietary hardware, can’t be replaced by off the shelf parts
Proprietary software, not open source.
Pricing

I’ve tried to come up with a build several times and hit a few roadblocks.

Firstly chassis: I have a 27U “half depth” rack, and finding chassis’ has always been a pain. I was able to find a nice 3U from Sliger for my gaming rig, and some 2U units for gaming server and Ripping machine, but never a 2-3U with any front hot swap bays that’s less than a mile deep. Max depth is 19”, can do 20" but cables get a bit smushed.

Secondly Software: I have tried TrueNAS, and I know there are many here who live and breathe it and I respect that, but for me the simplicity and ease of DSM is a comfort, and much more my speed. I’ve heard horror stories about Unraid, so I’m not sure that is the way to go either. I’d like something I can load on an SSD or pair of, rather than usb.

Thirdly Power: I love the way Intel NUC’s and most Synology units just sip on power, and for a device that will be perpetually on that’s pretty critical for me. That being said, I don’t want to be constrained to the limits of the Atoms and Celeron’s most Synology units ship with. There has to be a happy medium somewhere, maybe with the efficiency of Alder lake or an AMD platform.

If it helps for suggestions the machine’s main tasks would be file service, mostly domestic use but I often need to upload terabytes of data at a time so fast R/W would be useful. It also runs a Jellyfin container serving 10 users, so more compute the merrier. Also PCIE 8/16X slot for transcode GPU wouldn’t hurt.

1 Like
2

Hmm, i was about post something similar as above, so good timing. For the similiar version thought process as above, i decided to build custom truenas build 5 years ago, and I am now planning refresh.

Why? I was not that experienced then and it came to bite me in my ass, especially expandability. Quick check with current market shown me, that I cannot buy better hardware for similar or even double price.

My personal observations:

  • nice small form factor with external 3,5 drive bays is next to impossible to get without going to nas vendor
  • all purpose build nas are either extremely weak for the price, or just extremely expensive
  • if you do DIY and are content with 4 x 3,5 drives, there is only the supermicro chassis that IXsystem uses (link)

DIY build:

  • embeded consumer grade cpus (celeron, athlon, xeon-d, epyc embedded) are very weak, extremely cutdown in capabilities and on upper ends still expensive
  • lower end server part used to be relatively cheap and offer much better performance than alternatives ( ex. xeon E3 platform)
  • beware pcie line limitations and thoroughly check motherboards specs with future possible explansion in mind.
  • scan through you local second hand market, there might be high end server hardware parts available for peanuts now.

Current NAS build:

  • Supermicro x11ssh-f ; 2017 for 250€
  • Xeon E3 1225 v6 4C4T skylake era ; 2017 for 235€
  • 2x16 DDR4 (non ECC though)
  • 1x32GB system SATADOM drive
  • 3x16TB toshiba enterpise MG08 drives in raidZ1
  • Fractal desing node 804 case - no drive cages :frowning:

Shitton of storage that can be expanded, no performance bottleneck. Can host apps via k8s substystem and does host emby, homeassistant and some monitoring.

Wanted to add addon carrier board with 4 m.2 drives and realized that this thing does not have x16 slot, and even if had, it would not have enough lanes to be allocated. Bummer.

Power consumption at wall @ 240V:

  • start power ~~90W
  • off power ~~ 3W
  • idle ~~33-40W (drives do not spin down)

Since this thing runs 24/7, idle power is still reasonable and most of it comes from drives. Even if I pay .32€/kWh.

While atoms might have lower tdp, my gut tells me they would waken to much high power stale and remain awake longer, nullifying potential efficiency gains.
And these boards have doubled in price since covid, i.e replacement board+cpu would cost double I had paid for existing board+cpu. Its pointless to contemplate further.

That and even the best reasonably priced xeon-d has half or less performance than this old xeon.

Candidate new build:
I managed to snag used xeon gold 5118 for about 40€ .

While its single threaded performance is worse , it has massive expandability potential (48 PCIE lanes vs. 16) and much better memory subsystem - hexa channel architecture.

Getting memory on cheap is possible and surprise surprise, using udimm or udimm ecc modules is not possible. 16GB rdimm module can be bought for 25€ per, so not bad price for full kit of 6.

Board are hard to come by and expensive new, there is no second hand market and foreign used market is very expensive as well (minimum +30% price surcharge due to VAT and duty).

So new board will run between 500€ to 780€ depending on size and config.
I am thinking of going with either:

On plus side, lot of feature that would have to realized via add-on cards are already integrated, like 10GBe intel networking and mini-sas HD ports.

3

The hardware requirements really differ from what you require the system to do. If you are running only 8Tb of raw storage, I would guess that a low end core or ryzen CPU is fine. The biggest problem that I have ran across is the limited PCIe-lane count. Most of the system nowadays have either 20 or 24 usable lanes. And 16 go to the primary PCIe x16 and the primary M.2.

For context, here is what I am running on Truenas Scale

image
image1670×839 188 KB

I hid the IP:s because they are not relevant to this case

CPU usage

image
image1575×543 65.2 KB

And the applications

image
image1642×886 248 KB

4

TLDR: /s Go away with your reasonable arguments and well thought out responses. You undermining my VISION here :slight_smile: /s

My projected needs require expandability AND similar performance level AND non- hacky ECC support.

Expandability is the keyword, because it bit me in the ass hard - so I will risk going overboard this time. What I really want is to add multiple nvme devices and have additional lines open just in case further expansion, and that extremely hard thing to do if not taken care at planning stages.

This eliminates all consumer build candidates, and cost wise it also eliminates modern workstation/server due to sheer cost. I can also forget about threadripper and threadripper PRO build due to the same.

I used to plan around lower end epyc, but nothing is available and I dont want deal with OEM lock from second hand market.

Epyc embedded boards are weirdly overpriced, so no-go too. I.e there was +100% jump since two years ago for the same hardware.

EDIT: Fooling around with server hardware is a bonus. I dont really ned 12C24T, but its what I got cheaper than m,y 4C4T. Snagged this for peanuts, so either I use it for the build or keep as tacky paperweight (should be working but unverified):

MicrosoftTeams-image
MicrosoftTeams-image1247×1663 140 KB

1 Like
5

The world is moving towards m.2 storage, so unless you need to go big storage a 6 bay 24 TB m.2 NAS would cost you somewhere around $1500-$1600 drives included, 30-40W consumption.

Of course, if you need more than 24 TB m.2 does not cut it yet, wait for 16 TB SSDs to reach $200 and your drives to reach EOL before you throw away the spinning rust.

1 Like
6

If you aren’t limited to micro-ATX, a 16C/32T EPYC 7302P with mobo and 64gb DDR4 ECC can be had for under $500 from ebay. That’s for a PCIE Gen3 mobo. Gen4 boards are $100-300 more. If you’re staying with your current chassis and require microatx, the EPYC value proposition is tougher, as you described.

Also, not sure what it’s like for Xeon Scalable, but EPYC systems use more idle power than their consumer counterparts. For example, my EPYC 7532/Tyan S8030 with 128GB DDR4 2666 idles right around 50W, compared to 40ish for 7900X/X670 and 64GB, and ~27W for a 13600K/W680 and 64GB. These are readings from the wall with a single M.2 boot drive for testing. The increased idle draw and massively reduced single core performance was worth it for me because I traded spinning hard drives for U.2 NVME and ended up with more or less the same power usage +/- 10% or so.

7

Capacity end goal is somewhere in the neighborhood of 50TB+, so 6+ 3.5" drive bays. I thought about SSD’s but cost is still not quite within my reach for that. I’d like to just continue adding more 12TB drives since I’ve already got the three of them

8

What about something like this enclosure, with a X670/B650 micro atx board and a 7600 (non-X to keep the power/heat down). Fairly compact, with 8 hotswap SATA drives, and takes micro atx so your motherboard choices are better than with itx. And ECC if you want it (and is supported by the motherboard). Of course, heat is going to be a problem, as is finding a half-height gpu that can transcode, although I’m pretty sure some older quadros are probably available in half-height.

Or something similar with like a 13400, which gets you quicksync on the plus side, but you lose ECC if you want it and your performance with the thermal constraints of this type of chassis will probably be worse than with Zen4.

Either of these choices will have gobs more compute than your current Synology.

1 Like
9

I really like the X670/B650 mATX and 7600 idea, ECC is definitely a want so that’s good to have. I’d like for the chassis to be rackmount, I find them easier to cool and work on being able to pull out on rails.

I was thinking older Quadro myself for gpu, but it looks like there’s a few 10 series that are also half height
Edit: Gamer’s Nexus recently mentioned an Asrock A380 in Half height, maybe intel Arc time? :thinking:

At this point looking the cost of Icydock and supermicro 5.25" to 3.5" docks, and backplanes, it might make more financial sense to buy a Dell Poweredge R720 for 100$ and cut the rear 1/3 off, make a custom rear plate, put motherboard mounts in and go from there.

10

I commend your DIY enthusiasm, and I would absolutely subscribe to that build thread!

1 Like
11

I had to do that to a HP tft-7600 KVM console, the rails were nearly a foot too long lol. The convenience of a half depth rack is often highly inconvenient for mounting solutions.

most of my in-laws are machinists so I’m fortunate to be able to bring them models and plans and receive professional help for free, and in return I do all their IT support needs haha.

Will definitely spin up a thread if I end up going that route, there’s a few chassis that fit my needs but I’m not about to pay 700$ for 100$ in steel and a hot swap bay.

1 Like
12

Understood. Then there are two options, really.

Either do as LTT suggests and get a disk shelf. 3u, 12 disks, only needs a server on top of that which could be a 2u rack unit housing something that has ECC support. That is for if you are serious about storage. However, this will draw a bit of power.

As an alternative, my favorite el cheapo setup for a six bay NAS, drives excluded:

PCPartPicker Part List

Type Item Price
CPU Intel Core i3-13100 $136.99
Motherboard ASRock Z690M-ITX/ax $149.99
Memory TEAMGROUP T-Create Expert 2x16 GB DDR4-3200 CL16 $49.99
Storage Samsung 980 250 GB M.2-2280 PCIe 3.0 $33.54
Case Fractal Design Node 304 $109.98
Power Supply EVGA 500 GD 500W $64.98
Custom Orico m.2 6 port SATA RAID controller $49.99
Total $595.46

Drawbacks: No 10GbE or extra m.2 slots, no ECC support, only six drive bays, and only a single PCIe slot. For everything else NAS wise this thing is a BEAST! Awesome for home use.

I do want to warn you though that buying new HDDs will stop making sense in a very near future, 2025-2027 sometime - and even then, 12 TB is the smallest drive that makes sense to buy new now. So do be aware any infrastructure you buy now for HDDs will need to be re purchased at latest around 2030.

Good luck, and stay clear of SMR!

13

Goddamn, these chinese prices are tempting, even with 30% vat + duty surcharge minimum.
Looks like I got paperweight after all, since I can buy memory+cpu+board with three times the performance cheaper than board alone :).

Any idea if this seller tidbit actually works?

For buyers in Europe with orders over 150 Euros, we can provide an effective low-tax plan. (You need to choose DHL expedited shipping, we will send the package through DHL, the declared item value is 25 US dollars each, you only need to pay about 5 US dollars each, and you can experience DHL's expedited logistics service )

14

thanks for the suggestion! I’m leaning more towards an AMD variant of your suggestion to get ECC support, but otherwise it looks like a good starting list to play with. The 10GbE is a nice quality of life thing to have, and the M.2 slots I’m thinking maybe using for cache, so I may have to go with a board with more of those, we shall see.

My thinking with drives was that 4-6x12TB drives will be plenty for now, and gave me a place to start accumulating. My storage is consumed by a combined effort to both acquire digital copies of all the media my family and friends enjoy for media independence, and preserving existing photos, videos, and old family stuff (letters, 40mm film, etc.).
when the time comes to look at more room it will be much more reasonable to go full SSD by then is my hope, and then maybe I can do another upgrade to full flash array.

definitely been careful to avoid SMR haha, don’t need those issues.

15

OP I empathize with your quest.

I started out with a RockStar ‘whitebox’ (z800) build, learned the hard way BTFRS RAID 5 wasn’t prod ready.

Then I did ESXi on that z800 and did passthrough of HDDs to FreeNAS.

Then later I got some over kill eBay rackmount stuff and did a baremetal TrueNAS RAID.

Then the powerbill got overwhelming and I got a Synology DS920+
Your list of pros and cons pretty much sums it up.
Good thing with the DS920+ is its just enough to run a VM or two, 5 or more containers and lots of synology apps and truck along without issue. It fits my use-case very well.

For example here is my current heimdall:

image
image1145×468 101 KB

Except for pfsense, reolink and ntoppng all that stuff is on the NAS, and that doesn’t show other things like g-cloud sync, backups etc.

I’ll say this, it was a real PITA to do anything outside of “the norm” on TrueNAS, for example use plugins that were not common/highly supported and then even mind boggling a VLAN setup was elusive. I mean there is one obscure post somewhere on how to do it that lawrence tech even referenced (as there was nothing else at all out there).

Now that TrueNAS has a linux version, I’m very tempted to try again some day, but I’m really really liking how low power my Synology is, and how turn-key most everything is, and the things that are not turn key are highly documented (like Ubuntu). My only gripe is the price-to-what-you-get ratio. To this day its still asking a lot to get 2.5G ethernet or better, and decent RAM. Also your other point, inability to repair/proprietary parts- that will suck when it inevitably breaks down.

But the form factor, power use, ease of use- I have backups and plan on having a 3-2-1 setup pretty soon and it makes me game to just replace when necessary but stick to the ecosystem (very easy to restore from a backup vs. other systems).

1 Like
16

There are a lot of weird PCIe cards around, this would take care of that, if you are ok with no GPU for hw encoding :slightly_smiling_face:

Also, here is the reason why I think m.2 storage is the future;

A decent 4TB m.2 SSD is $180 today, 12 of those is $2160 for a total of ~$3k for 48 GB of raw storage (40TB usable on RAID6)

Compare with $200 for a good non-SMR 12TB HDD, and with the above package, roughly $1800 for 72 TB of storage (48 TB usable in RAID6)… It isn’t hideously more expensive now. HDDs are still better at the higher storage tiers but SSD is closing in FAST.

17

What seller is that? I purchased from tugm4470 on ebay and they were solid. I mentioned the thread over on the servethehome forums and got free FedEx shipping to the US, which got me my stuff in under a week (5 days from purchase to delivery in Chicago). Can’t speak to any of the other sellers, but folks in the servethehome thread have experience with some others.

18

I was opriginally looking at this bundle 7551P + H1SSLi + 8xRDIMM. from seller TUGM4470. As the excellent pricing suggests, its refurbished hardware from china.

Seller has excellent reputation, but i bough this one instead - 7302P+H11SSL-i+8x16.

1 Like
19

on the AM4/5 train of thought, It seems like ECC support is sporadic at best, with only the highest end X670E boards showing both support and functional error correction. disappointing to say the least. I really like the idea of a Ryzen 5 7600, but I’m not going to pay that much for an overkill motherboard just to get ECC. Considered falling back to AM4 since ECC support seems fleshed out on B550/X570, but I do really like the onboard graphics for troubleshooting, and as far as I can tell the only way to get both ECC support and iGPU is with a PRO -G part which are OEM only.

20

AM5 and ECC works fine on at least a couple of Asus motherboards