First Truenas Core build, suggestions needed!

Hardware Build a PC
, ,
1

Hi,

I’m building my first TrueNAS Core server and would like some help choosing some hardware.

Purpose of use​

  • Home NAS for my family
  • Mainly a media server (Movies, photos, office stuff etc.)
  • Only NAS functions, all applications, VMs, Docker, etc. run on a separate Proxmox server in the same rack
  • I don’t have good experiences with an “All-In-One” type server, so for this reason a completely separate NAS build.

Build goals​

  • As reliable as possible
  • Slightly overkill, but still “energy-efficient” build
  • Snappy experience when browsing and moving files

Current state of build:

The parts I have already acquired:

Case:

  • Logic Case SC-4324S - 24-Bay 4U case (6Gb/s backplanes)
  • Practically the same case as NORCO RPC-4224 4U

CPU:

  • Intel Xeon Silver 4112 (4c/8t)
  • According to the Servethehome review, this should be a pretty power-efficient choice

CPU Cooler:

  • Noctua NH-D9 DX-4677 4U + 3647 mounting kit from Noctua (Free)

RAM:

  • 6 x Samsung 32GB DDR4 2400 MHz ECC REG M393A4K40CB1-CRC (192GB total)

PSU:

  • Seasonic 650W PRIME TX-650

Parts still to be purchased:

Feel free to suggest hardware which is not purchased yet!

Used stuff is ok, but should be available in the EU.
I try not to buy anything from China because of the risk of a cheap copy.
I prefer server grade components.

MB

  • The intention is to proceed with the X11SPL-F. The motherboard was chosen mainly because of the number of pcie slots.
    I don’t think I need x16 slots for NAS use, or what do you think?

SLOG

  • I’m thinking more about the necessity of SLOG when the server is up and running.
    (If something seems to need sync writes)

  • I will probably end up with the Intel OPTANE 118GB SSD P1600X drives as they are relatively cheap. Probably in a mirror configuration using a Supermicro pcie bifurcation card. I’m still interested to hearing other suggestions!

Special VDEV

  • Can I use consumer grade NVME drives in 3 or 4 way mirror?

  • As far as I understand, special devices are normal VDEV (not a logging device), so the usage is not greater than with any other part of main pool? For this reason, there seems to be no need for an optane write duration?

  • This is perhaps the most interesting topic at the moment in terms of build progress. I would like device recommendations, as well as personal experiences. Adding a special VDEV would be the simplest when the pool is completely empty. I would prefer Supermicro pcie bifurcation card and NVME size format.

  • If you use Special device, what hardware do you use?

L2ARC

  • Can I use consumer grade NVME drives? Does the mirror bring any advantage? If I use special VDEV, I guess L2ARC is not needed? Otherwise, the idea would be to use L2ARC in “metadata only” mode.

  • I will think more about the necessity of L2ARC when the server is operational. If I end up not using the special device VDEV, L2ARC could perhaps slightly speed up the search for metadata (in metadataonly mode) from the rust vdev(s), so browsing large image folders, for example, could be more snappy. I’m still interested to hearing suggestions!

  • If you use L2ARC, what hardware do you use?

Pool

  • Start with 6 x Seagate Exos X 20TB (RaidZ2) OR striped mirror, which I can start with one pair of 20TB disks.

  • What is your recommendation?

  • 6 wide RaidZ2 and striped Mirror do not have a big difference in the usable space. The expandability of the Mirror and starting with a just one pair of disks would be attractive, because it will probably take me quite a while to fill even one drive. RaidZ2, on the other hand, would bring much more peace of mind.

Boot

  • SSD, SataDOM or NVME, which is your recommendation? Is Mirror useful?

  • My goal is to minimize cable jungle inside the case so using Sata SSD’s is not my first option. For this reason, Supermicro pcie bifurcation card or NVME in motherboard slot would be the priority.

  • The M.2 slot of the motherboard is connected to the PCH of the motherboard, does this mean that the M.2 connected to the MB slot does not reduce the available PCIE lines? If so, this would sound like the best option for the boot drive?

HBA

  • Would prefer a one 24i HBA, LSI SAS 9305-24i could be a good option?
  • I know the 8i HBA is sufficient for my current needs. The idea of 24i would be that I could use all the backplanes of the case at once and thus distribute the VDEV to them evenly. In this case, the failure of one backplate does not take the entire pool with it. Does this make any sense?
    Of course, this can be implemented with e.g. 8i HBA + Expander, but I would like to save pcie ports as much as possible. Believe it or not, I also think about power consumption. I would think a single 24i HBA would be more energy efficient than an 8i + expander.

10 GbE NIC

  • Will add later but will take suggestions

OVERKILL??

  • I know this build is a bit overkill for its intended use. (pure storage server)
    However, it’s about a hobby, not so much about how to get thing x done as cheaply and with as little power consumption as possible.
    The main components do not cost much more than consumer-grade significantly slower hardware. For example, the CPU and 196 GB of ECC RAM cost a total of 300€.

  • I live in northern Europe, where it is cold outside for more than half of the year. The server rack is located in an outdoor warehouse, which is heated by direct electric heating. During the winter, the use of the server is practically free, when the warehouse is heated at least partially by the waste heat of the rack. Electricity is also not particularly expensive in the area where I live, around 0.1€/kwh.
    Despite this, I have tried to choose components with as little power consumption as possible from the various overkill options.

2

I dont know about this but I think someone should stop you now and consider going for TrueNAS Scale instead, as iXsystems IIRC recently announced that it will stop developing for its BSD based Core.

3

Except that many even here have advised against it since it’s still beta quality a best. The FreeBSD base is indeed more solid if your primary use is for storage. That being said, FreeBSD 14.0 also works great if you can live without the UI :wink:

Ditch SLOG and L2ARC, they’re just going to increase complexity for no real benefit.
Not sure what you’re asking about “special vdev”, depending no how much you want to spend and how important the data is you have mirror, raid-z(1) and raid-z2 to choose from on the local machine.

Seagate Exos are probably fine, I would lean against Toshiba MG ones though…

A regular SSD will do fine, doesn’t need to be anything fancy but you might want to avoid the bottom of the barrel ones.

As for HBA, make sure cooling is adequate otherwise controllers like ASMedia ASM1166 will likely be fine even if you max out your spinning rust.

The motherboards you’ll be looking at will probably offer 10G NICs, Intel and Broadcom are likely the most common ones and wil do fine.

4

Do you mean the HDD in the link? The choice of Seagate Exos is not yet locked, so Toshiba disks could also be an option. They are at least a bit cheaper than Seagate disks.

geizhals(DOT)de/toshiba-cloud-scale-capacity-mg10aca-20tb-mg10aca20te-a2851607.html

Truenas scale can be an option, apparently there is an update coming soon where the ARC partition can be adjusted to match the core version. The “locked” 50% has been one of the biggest reasons to avoid using Scale, because in my case there is no need to allocate memory for applications, VMs, etc.

“special vdev” → Fast vdev which contains all metadata of the pool

5

You could check out the dual actuator seagate drives?