Help with a prospective NAS build

[Xyn]

Hey all. I’m currently running a Synology DS918+ with four 12 TB WD Red drives in RAID 10 since 2019. I’ve already had the first ever drive failure in my almost 38 years of living, so I’m somewhat paranoid of further drive failures, and in hindsight I wish I’d have gone with RAID 5 anyway.

I feel like the Synology UI largely gets in my way as far as services go (for instance, I had to modify the .mustache files to make ports 80 and 443 available for the LSIO swag container), so I basically host everything with Docker Compose anyway.

I’ve been considering a bigger and badder DIY NAS for some time now, but haven’t dug into specifics. When I heard about the Kickstarter for the UGreen NAS with a 40% discount for the first 48 hours, I went ahead and backed the 8 bay model for $899, as that seemed like a good deal at the time. Their software seems like crap, so I was planning to put Proxmox / TrueNAS on it, and that has me second guessing my plan the more research I do.

It is only this week that I learned that platforms like W680 exist that allow you to have both QuickSync, IPMI, and ECC support, even if it’s more expensive, as everything I’m seeing highly recommends ECC with ZFS. So I’m asking myself if saving $500, or even more, is worth losing out on ECC and IPMI.

The UGreen Kickstarter ends in about 37 days, so I have a little over a month where I can cancel my pledge without risk if I decide to.

So first, I’m asking for part suggestions. The main sticking point is the case. I’m planning to have 8 bays, and every 8-bay NAS chassis I’ve seen has at least one thing about it that pisses me off. Hotswap, good airflow, good PSU support, and such would be ideal. I’m also planning to put it on a shelf in the top of a closet, so it can’t be the tallest case in the world (it’s hard to measure up there, but I’d say around 2 feet or a little better).

Current idea:
Chassis: SilverStone CS382 - $249.99
CPU: Intel i5-12400 - 152.70
Motherboard: ASRock IMB-X1314 - 391.08
PSU: Corsair SF750 - 169.99
RAM: Nemix 128 GB (4x32 GB) Unbuffered ECC - 348.99
SSD (boot): Kioxia 256 GB PCIe 3.0 x4 - 14.99

So the current total is about $1327.74.

I will probably add on a couple of larger / faster NVMe drives in a mirror for caching at least, and if I can find a 10G / 5G / 2.5G / 1G card I’ll probably add that too. The PSU is also likely overkill, but it’s a PSU I would consider actually good so it’s what I have down for now, as I hate fan noise and want a quiet PSU.

I appreciate any hardware recommendations, or any ratings of this build. It will be my main storage server if I do this, and host a lot of services via Docker (currently, the plan is to have Proxmox host a Debian VM as a Docker host as I’m not exactly happy with the Docker implementation in TrueNAS Scale).

Likewise, I would like opinions on a couple of software approaches:

1. Proxmox with PCIe passthrough to a TrueNAS [Scale] VM
2. TrueNAS [Scale] bare metal with a separate machine like a mini PC or something to host Proxmox for VMs,

Option 1 basically lets me have an all in one device which I like. Option 2 should work as well, but would probably be more work and expense. With option 2, there wouldn’t be much need for QuickSync so I could probably do a Ryzen Pro build with ECC support.

If anyone has opinions on one approach versus another that would be helpful.

I’m sorry for how long and rambly this ended up being, but I’m under something of a time frame here and I’m a pretty indecisive person, so any advice or help is greatly appreciated!

[Jojo314]

Hey! cool plan, I have nearly an identical build running(sort of) in my rack:
Silverstone RM21-308
Asrock IMB-X1314
128GB ECC memory
corsair SF750
5x 12TB Raid Z1, 2x4TB Mirror Seagate Ironwolves

Synology 4bay 4x12TB for backups

All of those I can heartily recommend, especially the motherboard, absolutely loaded with features, the BIOS is really fully featured and easy to work with, and triple 2.5G LAN ports is AWESOME, you likely wont need an add in card unless you want 10G.

I initially went with a 12600k planning on migrating all my docker containers to this machine as well, but given the complexity and PITA of docker on Truenas Scale I decided to run just Truenas Scale for storage, get a more power efficient CPU, and move the 12600k to another dedicated dockerhost machine(maybe Proxmox cluster of machines eventually).

If you are more comfortable or happy tinkering, than don’t let me push you away from doing both on one machine, proxmox as a host seems a good way around the docker limitations within Truenas.

Here’s where things get dicey.

The lowest end Alder Lake CPU that supports ECC is a 12500. There are plenty of claims that the i3’s do, but there is no confirmation of this on intel ARK, and having bought and installed an i3-12100 yesterday to find ECC not working I can confirm that i3’s don’t support it.

In theory then the i5 12400 should support ECC, but from what I can gather online, yet again, it does not. if you go on Intel ARK and filter for processors with ECC support the lowest you will see are the 13500 and 12500. I’ll be ordering a 12500 this weekend and will update with ECC support results. Annoying since the 12600k is cheaper on sale currently, but i really need the power usage as low as possible for my machines.

[Xyn]

Cool. Can you confirm whether that motherboard supports IPMI or not? I would swear I saw IPMI somewhere when I was looking at boards on Newegg, but I don’t see anything mentioning it on AsRock’s site.

I think I may want the 10G add-in card. Currently, I’m all 1G and will have to get some switches regardless (I’d love to get the house wired for ethernet and put in a proper network with Unifi or Ruckus or something, PoE switches, multiple access points etc., but that’s a big project). The Iron Wolf Pro drives I’m planning to use have a max theoretical read of 285 MB/s, so with 8 of them in RAIDZ (not sure how I’ll set that up yet, but I’m thinking two groups of 4 drives in RAIDZ1 or RAIDZ2), that should give 8x read speed, or around 2280 MB/s, which is around 18.24 Gbps, so I think I could saturate 10G with it. At least, it should be 4 times what the 2.5G is if my calculations are right. Bummer the board doesn’t have triple 10G, that’d be killer.

Intel’s site sets the 12400 supports ECC, but I wasn’t necessarily settled on that one; I just saw Wendell talking about the 12400 / ECC / W680 combo in an old video in the last few days (I think that’s how I found out about W680 at all). The important thing is that it’s got QuickSync and ECC support, while having the horsepower when I need it and being power efficient at idle are bonuses. If I have to get a 12500 I don’t particularly mind. An extra $45-50 is nothing in the grand scheme of things with a build like this.

Yeah. I am not happy that TrueNAS Scale forces k8s instead of Docker Engine (and TrueNAS Core doesn’t support Docker at all). While the traefik / Ingress part of TrueCharts sounds good, I don’t know that I trust my services to them, so I’d rather stick to what is working, and that’s just pure Docker Compose.

I figured things would go smoother with a separate Docker host. Wendell made a video about using a VM within TrueNAS Scale to do this, but it seems like TrueNAS Scale doesn’t create the network bridge the way it’s supposed to, and I’d rather use Proxmox for VMs anyway, so that’s why I was leaning toward just using Proxmox and doing PCIe passthrough to a TrueNAS VM to give ZFS bare metal access to the drives.

I could get a Minisforum MS-01 or something as a Proxmox host though if the consensus around here is that that will work better.

[Jojo314]

It does not have IPMI unfortunately. A 10G add in card would definitely be of use, especially if you’re going to regularly saturate anything lower

If Wendell says 12400 supports ECC I’m happy to believe it, all I can confirm for certain is the 12100 definitely doesn’t lol.

you can definitely virtualize Truenas and run everything through proxmox if that’s your taste, i know lots of folks here do similar. I’m only moving my storage and services to separate machines for ease of tinkering for me.

[Xyn]

That’s a bummer. I may want a different board just for that, as I plan to put it in the closet and I really don’t want to have to unplug it and carry it over to a monitor and all that if something goes wrong. We’ll see. I do really like that Silverstone CS382 though, and it only supports up to mATX. So I don’t know that there’s a mATX LGA 1700 W680 motherboard with IPMI. I may have to figure out a different case and get a full ATX board. I really just started planning this a couple days ago, as prior to that it was always just a vague idea I might build one one day.

Having about 35 days to decide if I want to keep or cancel the pledge on that UGreen has put me on a clock though, so I’m trying to get a head start on figuring something out.