Energy efficiency for a homelab build server, or do I just go to the cloud?

TL;DR: please, do more energy efficiency benchmarking, everyone.

I pay $AUD0.314/kWh for power. It is very hard to find good info on what the most energy efficient CPUs and GPUs are, especially in relation to idle power and something like kernel-compiles-per-watt.

I’d rather not turn a server on and off all the time (although without spinning drives maybe powering on/off isn’t so risky any more?)

It’s at the point I either pay silly money for an M1 based system and some Thunderbolt storage, or I pay silly money over time to some cloud company somewhere.

31.4c*24*30 even in worthless Australian dollars buys a lot of cloud compute (and yes, I probably wouldn’t max it out all the time, but let’s go for easy maths first)

What would you do?

It depends upon the homelab, if you’re doing home serving of media it’ll cost an arm+leg to cloud that and with the amount of Intel NUC-like mini PCs(also off-lease micro pcs) you could in theory get by on a 45-95W machine.

If I couldn’t run my PowerEdge R330, I’d opt for a quad-core Mini PC as long as it has a NVme+2.5mm bay(quite a few out there with those specs) and have a power requirement of no higher than 65W. If I needed dual NIC, this Ryzen 7 3750H box comes to mind as it has dual 2.5 Gigabit ports:

Ok, forget serving of media - I have a slower Atom based NAS for slinging files. Good point on the Ryzen mini PCs. I wonder if anyone has benched kernel compiles on those?

There are benchmark sites of notebook CPUs which can give you some idea of level of performance, when it comes to mini PCs you’re really stuck with either Intel/Ryzen “H” or Intel “T” series CPUs if you’re aiming at best per watt performance ratios. I wouldn’t touch any Intel “U” processor for homelab usage as your mileage starts to vary in mixed usage cases.

As far as the Ryzen 7 mini PC, NotebookCheck displays a mix of benchmarks. You can also compare processors too.

Do you think the mini PC boxes are more or less thermally constrained than laptops?

What is your workload?

1 Like

Buildroot builds, armbian builds, openwrt builds, some docker image builds, the occasional Linux kernel build

Currently I run them on a 5800X in Hyper-V with 32-64GB of RAM allocated. I’d prefer to go to Xen or KVM, but I still use Windows for gaming.

Well, can’t measure my stuff at the wall but idle (well, as i type this - so at the desktop doing whatever background shit Windows 11 + Steam + GOG + Origin + EPIC + etc. does) power draw in Radeon settings (shows both CPU and GPU) for my system (5900x + 6900XT) is about 14-20 watts for the CPU and 10-15 watts for the GPU with out of the box settings.

Lets call it say… 50 watts when mostly idle (pessimistically)?

1kw / 50 = 20 hours to draw 1kh/h

So you’re looking at roughly 1-1.2 kw/h per day idle for the machine - assuming your numbers are similar to mine… mine should be a little worse most likely.

…or 31.4 cents = $9.42 every 30 days in idle power for the machine you already have. Assuming you don’t let it go to sleep.

Lets assume you’re working the CPU flat out at 140 watts.

That’s ~7 hours per kilowatt hour.

So you can run your lab CPU flat out for 7 hours for 31.4c. Without turning off PBO, power limiting, it, etc. And dropping its power limit or max clock would surely save more power than the impact to processing throughput due to the way power consumption vs. clock speed scales.

Ryzen 5k is one of the most efficient platforms you can get for compute per watt. You can bet that a cloud provider will charge you cost+margin for their power for the compute you consume. Yeah an M1 is good but you won’t pay off the machine with electricity cost savings in a hurry.

Unless you need the machine to be up 24/7 you could just have it sleep when idle?

Now those numbers above are cpu load only… but that’s going to the the vast majority of your compilation power consumption.

I just don’t think you’ll save money in terms of electricity by running stuff in the cloud. Just turn the machine off or sleep it when idle.

Cloud will however help if you need to run something permanently and have it accessible 24/7.

edit:
I think hardware unboxed and GN do efficiency ratings for CPUs.

1 Like

I’ve used a fanless Celeron box with minimal downtime(Windows Updates usually bogged it down), on the Intel or AMD “H” series a fair number of OEMs more or less use gaming laptop exhaust fans so noise levels could be an issue while mini PCs that use 35-65W CPUs typically have space to use the stock AMD/Intel HSF or custom low-profile HSF. Thermally I’ve had more laptops throttle if the heatsink was smaller than normal–for example my i3 8130U had a very odd thermal throttle, it would run at near full performance then throttle & wait for the heatsink fan to kick in.

There are some makers I’ve seen on Amazon use larger internal cooling for their Ryzen “H” series mini PCs so you have to browse a bit. I’ve only held off on any Ryzen mini PC as it’ll be interesting how Intel 12th gen mobile CPUs are used in mini PCs, if we see Intel Iris Xe level IGP instead of UHD Graphics it’ll be interesting for a mini server to have a faster QuickSync. Even the lowest end i3-1210U with UHD has six cores, 2 performance and 4 efficient which could be interesting–I feel the i7 having 2 performance and 8 efficient could have an interesting performance curve.