Advice on new Server

Hardware Hub Build a PC
, ,
1

I need some advice. Whilst I usually know what to do, I’m a bit stuck.

My current server has the following hardware:

Intel i3-8100
32GB of ECC UDIMM
Fujitsu / Kontron D3644-B Motherboard
1x 256GB Toshiba NVMe SSD (Boot) (ext4)
2x 2TB Corsair MP400 2TB placed in PCIe x4 Adapaters (with large heatsinks) (Mirrored ZFS Pool)
6x 14TB Western Digital WD140EDGZ / WD140EDFZ (RAIDZ2 ZFS Pool)
1x Intel X540-AT2 10Gb NIC
APC SmartUPS 750

The system runs Debian 12.

The server runs ~12 Docker containers for serveral home ‘purposes’ like downloads, home automation, private cloud stuff.
Currently the server idles around ~30w with the disks spun down.

I have a 4Gb Fiber connection, hence the 10Gb NIC.
The server is not busy continuously, but during peak hours it does struggle.

That’s why I’m looking to upgrade.

Things I’m considering:

CPU:
New CPU, since the current one is quite outdated.
I’m rooting for AMD, but Intel does have a much better lower idle power consumption, and with my usage profile, I don’t think I will benefit from the better performance/watt ratios for AMD CPU’s, but I’m not really sure.

AMD’s new 9000 series or the current 8000 series (G models) might have a fairly low idle power consumption which might make it doable.
I’ve looked at EPYC, but the idle power consumption seems just too high.

Memory:
AMD does have ECC support, but it’s… flaky… Intel is not much better, but there are some guaranteed options available with the W-series chipsets.

I know perfectly well that ECC is not a must for ZFS, but I feel a little bit better when using it. It’s just a matter of preference.
But as with all things, if the consideration is, ECC or a (much) lower idle consumption, I’d probably pick the lower Idle consumption.

Motherboard:
A new motherboard is one of the hardest choices. Going DDR5 ECC seems fine, but the number of motherboards with (proven / tested) low idle power usage are scarce. I’m not nitpicking on a few watts (~30W for the whole system would be fine), but I’ve read numerous stories regarding 50W+ idle consumption from several AMD and Intel motherboards, without any peripherals or addons. That’s just a waste of power (and thus heat).

SSD’s:
Nothing much to change, the seperation of the boot drive from the mirrored ZFS pool makes it so I don’t need to boot of zfs, which is much easier regarding OS Upgrades, maintenance etc…

NIC:
I’m considering using a motherboard with dual 2.5GbE onboard. My switch supports LACP, which might allow me to fully utilize my Internet connection without requiring a 10Gb NIC. Although I’m not sure the load will be sufficiently spread out of the 2 links when using Usenet or Torrents. Both use multiple connections, but it might not work properly…

Harddisks:
No changes required, just the note that the motherboard needs to have 6 SATA ports.

One more consideration, my server is placed in my Office, which is quite small (few m2), which makes managing the climate quite cumbersome. If the equipment in my Office consumes too much power, the room gets fairly hot during the summer, which is quite annoying.
So a lower idle power consumption not only saves money on the electricity bill, it also lets me keep my sanity :slight_smile:

So a little more investment for a lower idle power consumption is quite fine.

Can anyone recommend a good setup?

Budget is not really a thing, if you can guarantee me that it will idle around 30W, give me (a lot) more performance than my current setup, I would honestly don’t care if it costs 2k EU.

2

When you say at peak hours it tends to struggle, how so?

3

You will have to switch generations up to at least 12th since that’s the first with official ECC support on Intel’s low power options.
I did this in another thread, but for simplicity sake: your current vs 12900T

Intel Core i3-8100 @ 3.60GHz vs Intel Core i9-12900T @ 2.10GHz

Description: Intel UHD Graphics 630 vs Intel UHD Graphics 770

Socket: FCLGA1151-2 vs FCLGA1700

Clockspeed: 3.6 GHz vs 1.0-1.4 GHz

Cores: 4 vs 16
Threads: 4 vs 24

Performance Cores: 8 Cores, 16 Threads, 1.4 GHz Base, 4.9 GHz Turbo
Efficient Cores: 8 Cores, 8 Threads, 1.0 GHz Base, 3.6 GHz Turbo

Typical TDP: 65 W vs 35 W
TDP Up: 65 W vs 106 W

Cache per CPU Package:
L1 Instruction Cache: 4 x 32 KB vs 8 x 32 KB + 8 x 64 KB
L1 Data Cache: 4 x 32 KB vs 8 x 48 KB + 8 x 32 KB
L2 Cache: 4 x 256 KB vs 8 x 1280 KB + 2 x 2048 KB
L3 Cache: 6 MB vs 30 MB

Memory Support: Max. Memory Size: 64 GB (DDR4-2400, ECC Supported) vs 128 GB (Up to DDR5 4800 MT/s, Up to DDR4 3200 MT/s, ECC Supported)

CPU First Seen on Charts: Q4 2017 vs Q1 2022

Overall Rank:
1599th vs 299th fastest in multithreading out of 4757 CPUs
1103rd vs 136th fastest in single threading out of 4757 CPUs
464th vs 97th fastest out of 1368 Desktop CPUs

Multithread Rating

6111 vs 30093

Single Thread Rating

2218 vs 3818

1 Like
4

Welcome to Level1Techs!

Check out Wolfgang’s channel for resources and advice on building power efficient home servers.

Quite impressive!

You’ll find that this is quite hard to achieve with the list of peripherals you have (HBA, drives, NIC, memory).

In modern systems the components drive the idle power consumption more than the CPUs. It’s not only the PCIe cards and drives, but you also need to budget ~5W per memory stick.

Not only for the reasons you mentioned, but also from a power consumption point of view.
Full featured motherboards can come with many PCIe slots, fast and many USB ports, thunderbolt, multiple NICs, PCIe gen 4/5, etc. All of these things are great features that offer flexibility and make a mobo desirable from a functional point of view, but they also consume power, most even when idle.
For reference, I have a full featured AM4 mobo that with just a two ram sticks and a single NVMe drive idles > 50W, but I also have a AM4 mobo that in the same config idles < 20W.

Unfortunately, there is little reliable information out there about power consumption of motherboards. Wolfgang’s channel points to some but it mostly contains older gen hw.

I find that as a general rule (that may be broken!)

  • smaller mobos consume less power (ITX, mATX), because they generally have less features
  • newer PCIe gens require more power than older ones
  • CPUs are really power efficient nowadays (<1W idle), pick any desktop CPU you like and need for your workload
  • chipset (again based on features offered) can make quite a difference. Less is more from a power consumption point of view
  • Less ram sticks consume less power than more ram sticks. Luckily the current desktop generation has trouble with more than 2 sticks of RAM (AFAIK both Intel and AMD) so you’re good up to 96GB RAM. For performance reasons you want 2 sticks to utilize the dual channel memory capabilities of desktop systems

If your storage setup works for you from a performance and capacity point of view I wouldn’t change a thing. It’s obviously working very well from a power consumption point of view.

Good luck finding one. If you do, I assume it’s not a power sippy mobo.

5

The load of the system peak to 8+, with the CPU being loaded during PAR and unrar/unzip operations.
With 400MB/s data coming in, the CPU has to handle the network traffic, run the applications, yEnc, extract and move the data a few times. Also it has to do the ZFS stuff. The filesystems are not encrypted nor compressed.

6

| This should have been a reply to trytwicemedia , but this forum software is retarded.
Generally speaking -T series CPU’s are just power limited for P1 and P2 power limits. They’re not necessarily more efficient.
But, nevertheless, there are posts online that a 12900 can idle around a few watts, which would be nice indeed.

7

Yeah, the motherboard is the hardest part. I know peripherals are the issue. Usually because they mess up ASPM or other power saving stuff. I have an Asus Thin mITX board that I was attempting to run as a Router. The thing used ~3W idle with an i3-13100 with the onboard Realtek NIC disabled. When enabling the NIC it went up to 7W, because ASPM got messed up. When disabling the NIC, and adding a NIC to the M.2 slot with an adapter, it went up to 11W, because ASPM got even more messed up…

I’ve found this magnificent piece of equipment:
ASRock IMB-1714
It basically has:
3x2.5GbE LAN, 2x M.2, 8x SATA, 1x PCIe x16, 1x PCIe x16 (x8 electrical), 2x PCIe x4, 3x PCIe x1.

But who knows if it’s power efficient…

Oh, and I’m aware of Wolfgang, good channel!

1 Like