Low cost, high efficiency server refresh. Searching for best options?

Hardware Build a PC
, ,
1

Hello people, I’m starting to look into (after 3-ish years on my old solution) if there might be a better option to refresh my home server with. This seems to be the best place to ask for some advice!

My current solution is an older dell workstation (xeon E5 v1 age), running freebsd and a storage pool across 2x4TB hard drives. It’s been good over the time I’ve had it, but was originally bought to use the high processing capacity to run a lot of server software, game servers in particular. I don’t do this any more, running little software beyond samba and ssh. Energy prices are climbing without an end in sight, and I’m starting to feel curious if this now means I would save money overall by moving my storage drives over to a new machine.

Here is the list of priorities that currently seem important in any new computer:

  • Can run FreeBSD and ZFS (I’d really like to port most of my current configuration over, and spend minimal time setting anything up)
  • High performance/watt ratio. Actual performance only needs to be enough to saturate read/write over gigabit ethernet
  • Stable for long term use (should any replacement also use ECC ram?)
  • Supports at least 3 SATA drives (2x storage, 1x OS)
  • Very low cost (I have a really limited budget, and don’t think there is much point in a long buyback time as my needs may change)

So working from that, I think there is a couple of options (new builds are probably too expensive);

  1. Find a much more recent second hand office desktop, probably 8000/9000 series intel. Depends on the power usage of these in comparison to my current computer, and would be giving up ECC.
  2. Use external drive cages to open up more options for computers. Are USB3 external interfaces reliable enough for long term use? If using those, perhaps the raspberry pi 4 is a reasonable option, if it’s reliable enough (and ECC is not important)? Other computers possible may be the M1 mac mini (while far over budget, I would re-work the software to host files from macos, while using as general computer), or other NUC-type computers. I also have access to a 2012 mac mini, which might represent an improvement in power usage, with only the drive cage costs.

I’m waiting on a mains power meter to determine the actual costs of my current server, but thought it would be a good idea to gather information on other options in the meantime.

If anyone has other suggestions, or comments on these options please do let me know!

1 Like
2

I wouldn’t have recommend pi4 for drives, but that’s only because I like keeping my storage “encrypted at rest”. Other SBCs like Odroid n2+ or Odroid m1 (any many others) are a better pick for simple file serving IMO.

A pair of drives over usb3 would work fine, … maybe even several - I’d maybe look into 4 drive enclosures.

m1 mac mini is both stupid expensive overkill for 1gbps samba server use, and has a really short track record on asahi, which is what would give you zfs (unless maybe you ran Linux in a VM but that’s weirdly hacky, maybe in 6-12 months it’d be worth reinvestigating if you need a more powerful CPU).

Expect about 5W per disk and 2-7W for your SBC if you go down this route. Don’t expect you’d be doing lots of processing - but you might be able to run some other simple network software in a container or two.

1 Like
3

I once measured a mac mini 2012 with external drive enclosure (4x 3.5") acting file server at 31-35W (yes including the enclosure disks). USB3 is fine if it’s in a cat-proofed location but that also depends on not putting too many devices on that*, and a half decent external cage may be harder to find than you would presume. Boot/OS from internal (or a USB flash) and you only need a 2 bay.

I don’t think I’d say I’d recommend it, but for a DIY NAS from parts you predominantly have it meets low outlay & ongoing cost.

Trade off - low power, but you could still run low demand VMs on it (I did). If you can get FreeBSD on it native, it won’t be standard. I ran OpenZFS on OSX (then), but the mini 2012 is not supported by new OSX releases now.

And if you ever want much more from this than a file server - you’re looking at another machine, that’s another setup cost. I’d consider what you want from the next 3 years.


* (example, on a newer lenovo device I can't even use the *mouse* when copying to a usb3 SSD)
1 Like
4

How much money is very little money?

A RPi 4 + 4 bay NAS enclosure sets you back roughly $300 with an average draw of less than 20W.

Otherwise, here is a budget build based on a 10100F:

PCPartPicker Part List

Type Item Price
CPU Intel Core i3-10100F $68.08
Motherboard MSI B560M PRO $84.99
Memory G.Skill Aegis 2x8 GB DDR4-3200 $49.99
Storage TEAMGROUP MS30 128 GB M.2 $16.99
Case Fractal Design Node 804 $137.99
Power Supply EVGA SuperNOVA GA 550 W 80+ Gold $59.99
Total $418.03

Notes:

  • 12100 would be ~$50 to $100 more expensive but is current gen
  • 10100F requires a discrete GPU for installation, but can run headless after that
  • Intel does not support ECC RAM
  • 16 GB is probably not enough for ZFS as it stands
  • Case is expensive only because it supports 8 HDDs
3 Likes
5

I’m going to assume that you missed the “Can run FreeBSD and ZFS” and posted a rather generic reply. The RPi4 is a very poor hardware choice in general, no hardware crypto, 1x PCIe bus and runs poorly on any other OS than Linux because of lack of documentation available and/or overall interest. The MSI board you posted uses a Realtek NIC which is less than stellar in general and driver support leaves a lot to be desired. If you’re unsure of the answer at least mention that.

@EleaOwl
I’m pretty much in the same situation (finding new hardware) as you and right now there isn’t much of a “replacement” in general. You pretty much want as much as possible to be Intel, you can go with AMD Ryzen but be aware that most motherboards uses Realtek NICs (milage will vary) and ECC is “experimental” at best. Given your budget I think it’ll be hard to find anything that’s worth the money at the moment especially if you want ECC and given “recent” CPU vulnerabilities I wouldn’t recommend getting anything older than 11th unless it’s very cheap.

You can go ARM (aarch64) using Pine64 RockPro64 which works well in general (arm/RockChip - FreeBSD Wiki) but processing power gets it’s a big downgrade and 4Gb RAM will be a bit of a pain if you want to use ports and ZFS (expect to use 1-2 jobs at best in many cases). There are reports of using ASMedia SATA (AHCI) controllers fine but there’s less information of the newer 4/6-ports ones. I have one myself and it appears to work fine apart from a cosmetic issue (259705 – ahci: Asmedia ASM1166 lists 32-ports on 13-STABLE) but there haven’t been any long term testing.

If you want to use ECC your best bet seems to be finding a “workstation” using Intels W580 chipset (if you want 10-11th generation CPUs, W680 for 12th gen) but those are in general still sparse and quite pricey. Avoid USB for storage, it will give lots of weird issues that’ll be hard to trackdown and/or fix.

1 Like
6

Thank you everyone for your replies!

Lots of options to consider, although my final budget will probably be set by the combined value of energy saved (at it’s current rate, although that will probably climb) and a very small resale value of any excess hardware (retiring the old server would provide a very good spare machine, probably allowing me to sell off all of my older ones).

It’s interesting how opinions vary about usb interfaces for drives. Do any of you have links to (or descriptions of) more in-depth explainations of why/why not to use them for NAS drives? I’m impressed by the potential of the 2012 mac mini - since I already have that, it’s attractive to use and would not represent a large performance loss as it has 16GB of ram. I’m sure running freebsd would be comparable to how persuading it to run the latest macos was, which only took some time and careful reading.

The alternative SBCs are certainly interesting, although do feel to be on the expensive side, given that none of them have more than one native sata port, still requiring the usb interface (although, the rockpro64 may be interesting depending on the costs of used sata interface cards).

I don’t currently do any drive level encryption due to unattended power on and off of the current server (to save power), beyond software like sshfs, am I likely to benefit from any increased processing power, or hardware crypto (I use pkg instead of ports, and cross compiling may be an option if required)? Main reason to have used the pi 4 as an example is that I have one running some unimportant services elsewhere which could be re-used if it would be a good solution, however would absolutely look to other boards if they are better options and within budget.

Recognising that the m1 mac mini is overkill expensive, it does appear to not be much more (8GB model, second hand) than a new build (not counting the cost of a drive interface), and would be used for far more than a server (work computer, possibly HTPC). That option would represent a complete re-work of the software setup, and probably using a much older machine for regular backups (automated start up and shutdown) due to lacking ZFS.

I do have a few parts that can be re-used if they would make a new build more viable; an older intel gigabit nic, case with enough drive slots and carrying forward the current OS drive

1 Like
7

USB is flakey, many USB to SATA controllers are craptastic at best and ZFS will tell you about this frequently.

Regarding SATA cards ASM-based ones are usually quite cheap however quality isn’t consistent. I used this for https://www.silverstonetek.com/en/product/info/expansion-cards/ECS06/ for hooking up a bunch of optical drives and it worked fine apart from the iteration issue. Not sure if ASM1164 such as https://www.amazon.com/StarTech-com-SATA-PCIe-Card-4P6G-PCIE-SATA-CARD/dp/B09KCH846V exhibits the same issue

The RockPro64 is 79$ so not bad at all considering what you get, ROCKPro64 4GB Single Board Computer - PINE STORE also works very well as a router running FreeBSD.

3 Likes
8

I would go with a Ryzen 2400ge pro for the min or Ryzen 4700ge pro for the max both Tdps are 35watts so power efficiency is there and you could get a nice b-450 board since you said ecc isn’t a issue.

9

> Can run FreeBSD and ZFS
> High per/watt
> Only need to saturate gigabit
> 3+ SATA
> Very low cost

Look no further than a RockPro64. @diizzy knows his things about FreeBSD, I would listen to his opinion on this one.

FreeBSD was among the few OS that worked on my RockPro64, I tried with 13. Pretty neat. All you need to do is get a 4 or 5 port SATA PCI-E card (but not the official pine64 one, as that one is not compatible with the rockpro64) and either pine64 official rockpro64 case, or better yet, just grab a Chieftec CMR-3141SAS or any Icy Dock or StarTech 3x 5.25" to 4x 3.5" bays and a pico-PSU and you’re set.

It’s small, barely uses power, it’s cheap, runs freebsd and zfs, and should be good enough for gigabit speeds.

If you want something with more oomph, just grab a used optiplex, maybe at least 8th gen intel.

I’m a fan of Odroids in general, but it seems amlogic support on freebsd is a bit lacking. Besides, all you get are those:

Used to have more support for other boards, like banana and cubie

2022-08-16_00-08-1660599684
2022-08-16_00-08-16605996841046×371 46.7 KB

I would not go with USB. Many converters do not pass SMART data and other things that ZFS kinda needs to make sure drives are fine. Can’t go wrong with SATA.

Just my experience with running a USB m.2 SSD enclosure on a RPi. No TRIM, no SMART. The poor drive has to be suffering.

Get a small UPS for a SBC.

That’s not a bad option. Maybe even athlon 3000g, should be plenty for what you need. Not sure what options would support ZFS, but ZFS is optional really. I’ll be running ZFS on Odroid HC4, but unfortunately, again, no FreeBSD support for amlogic. Besides, even other distros are having a bad time with ZFS on aarch64, as I have discovered. So, rockpro64 is probably the safest bet on the cheap that doesn’t use a ton of power. But Ryzen should probably sip power at idle too, so should newer gen celeron, pentiums and i3s.

Actually, I used to have an ASRock J3455M with a celeron quad core (it had atom cores), you can probably find something similar with a gemini lake or newer celeron, it should not break the bank and it is similar in power consumption, but still gives you x86 and some built-in stuff, like 3 sata ports. But it’s not the best bang for the buck, but going that low-end, you probably do not even need to consider that, as the price itself jumps quite a bit.

10

Or LattePanda3

1 Like
11

They’re still “supported”, just that no images are generated.

1 Like
12

According to FreeBSD wiki, the RPi 4 still runs FreeBSD and therefore ZFS. I agree it is not the most amazing NAS of all time, but at < 20W footprint and $300 it is the most low power and definitely cheapest new HW option that I know off.

Of course if your budget is $300 or lower all your options will be pretty much terrible. If you want all bells and whistles with 32 GB ECC and a six core CPU, you will land on $600-$700 bare minimum with a Ryzen 5 5600G.

True, I put together that build just looking at Cheapest-but-atleast-decent-quality angle. As always the build above is a starting point, feel free to modify for parts with better compatibility. :slightly_smiling_face:

3 Likes
13

BTW if you’re looking for M1 Mac Mini, I’ve heard good things about these enclosures:

Specifically that they have a decent set of chips inside - a friend is saturating 10Gbps ethernet with these.

What’s the plan for the software stack for ZFS on your M1 Mac Mini? … TrueNAS in a VM?

14

It doesn’t list anything about PCIe or HATs though and there is a warning about RPi4 here: arm - FreeBSD Wiki

Again, your suggestion doesn’t make any sense as the 5600G does not support ECC. You’re need PRO CPUs for official ECC support on Ryzen.

1 Like
15

If ECC is a hard requirement the Ryzen 5650G PRO costs $20-$30 more than 5600G. So technically correct, but not really that important at the planning stage.

Yes I confused ECC support with 5600X.

3 Likes
16

I’m a little surprised nobody has mentioned the Seeed Studio reServer [sic]. It comes in different versions, which are all listed in the Specifications chart at the bottom of their webpage, for comparison.

It’s x86_64 based, and so you don’t have to worry about whether it’s supported or receiving updates. Since you are using 2 x 3.5" HDDs you need something which can accomodate them inside the same chassis, and the reServer can. Also, I have their Odessey Blue which I’ve been running OPNsense (freeBSD) on with zero problems. They’re both based on the same design, and should be equally compatible. The affordably-priced i3 version would offer power savings, but I’d go with the 4 core version if planning to run VMs.

Oh, and the networking is dual 2.5 Gbps, which is a nice bonus. It could even work as a nice router. Also, supports WiFi and BT.

It sounds to me like you should try TrueNAS Core, which is a freeBSD-based NAS server distro, which has been fine tuned for great performance. It has built-in Samba and NFS, iSCSI, etc. It can even do BSD jails using Beehive. Sounds perfect for your current use.

Of course, it’ll probably also support TrueNAS Scale (Debian Linux), which has even better virtualization, but with slightly slower performance, particularly with random 4k.

1 Like
17

You could replace this with a b450 or x570 plus ryzen 3400g, saving power and improving performance. V1 e5 xeons really are quite old now.

If you want to maintain ECC you may need to swap the cpu out.

18

saying as your not doing any high end compute and your just wanting access to storage?.
grab a ryzen 3, a micro atx board and stuff it with as much ram as possible and call it a day.

power draw will drop about 80%.

or pull one of the cpu’s and under volt the other.
old sandy bridge didnt mind running at 1.125v or less.
and could reduce power by 25%. (in fact i would try this now, just to see if you can get away with it)

2 Likes
19

Yeah the only reason i suggested the 3400G (Zen+ based) is because its fast enough, cheap and needs no discrete GPU.

1 Like
20

Better yet, just get an Asrock Deskmeet.

https://www.asrock.com/nettop/index.asp#DeskMeet

There is an AMD Ryzen version, and you could just throw a cheap $50 Ryzen G series in there, and set the UEFI BIOS to “Eco mode”. This could drop the TDP to only 35 watts. It’s cheaper than the Seeed Studio reServer I mentioned earlier, but it’s larger and doesn’t include dual 2.5 Gbps networks. Although, the Deskmeet supports add-in cards, and that could allow even 10 Gbps with an affordable card from TP Link, or the like.

I actually have their Deskmini, with a Ryzen 3400G serving as a Proxmox node. In the past, I’ve also installed TrueNAS on it, with mirrored 2.5 inch drives. Worked great for that, as well.