I am just looking to see if there is anyway I could bring down my energy usage. I currently run a Dell T440 (Dual Xeon Gold/256GB ECC RAM/ 6x10TB enterprise drives/dGPU/10GB NIC)
Server is running TrueNAS and use it for containers (plex, jellyfin etc:) bunch of VM’s for testing applications I develop and file server.
On average it uses about 6.5KWh per day. Just wondering if there is any better hardware options I can switch to in order to save power?
I things I can’t compromise on is storage capacity, I may need to add a bunch of extra drives in the future to expand storage and also remote management of server hardware.
Something a lot of people seem to struggle with and it’s getting worse as time goes on.
Depends on what you want to do.
Why do you need remote management? At best, you can do a pi-kvm add-on (so you need to budget that in the build), but besides SSH, I don’t see a lot of reasons for IPMI on a home setup.
HDDs will always use more power, about 6 to 8W per drive (some go even higher). 6 * 6 = 36W on a good day for the drives alone. Add a computer and you can probably go in the 80-100W range. I suppose the reason for storage capacity is your media library, which I guess you can’t really do without.
Have you considered not keeping the thing powered on 24/7? Does it really need to run when you’re not at home or when you’re sleeping? As a starter, start turning it off and have VMs automatically start up on boot.
Then as a low-power option, if you can afford it, you can’t really go wrong with the newest ryzen 9000 series, these were made to use less power than previous gen. You might want to look for motherboards with good pci-e bifurcation support, if you can’t find one with 10G on-board and with enough pci-e slots to allow you to connect SAS cards and the GPU.
You might also want to consider a tiered setup. You could go with something like a ryzen 8600G or 9600x and keep your setup small for your “essential” services (like jellyfin), but keep the dell server around when you need more oomph or want to play with a lab (just power it on remotely wait 5 minutes and you’re good to go). That’s what I’m doing with my threadripper 1950x (since it’s kind of a power hog compared to my i5 11500T thinkpenguin system, or especially my odroid h3+ or rockpro64)
Important question for power cost is looking at the load. What is the average load/idle load on your servers?
6,5kwh per day is 270W constant power use, probably because of the high base power consumption.
For comparison, i have a fileserver running at 33W with 3x16tb+2x4tb.
Looking at your system, the dgpu, 10gnic and main platform is the biggest draw.
Where do you have the ECC requirement? You would need an AM5 board with a ryzen pro to run ecc, or a threadripper platform. Or you can split the fileserver apart from the virtualisation.
Looking at that you need it for work, a more efficient single cpu epyc system could be the best fit. a dgpu, lots of hard drives and a 10g nic will require pci-e lanes.
Its important to look at actual load.
If you have high load then you need to match it to what performance you require.
If you usually run at low load or idle, the actual power or performance of the cpu doesn’t really matter and the efficiency of the motherboard and power supply is much more important. a dual cpu socket system would be the worst idea for this.
how many threads are u using. if that is not all of them. How do’s the system perform on one cpu ?
And maybe put the things you want to run 24/7 on somthing like a n100 or somthing else really low power. And have this beast only running when you are in developing mode
Not quite true… For the APUs, yes, you need that. But most AM5 regular CPUs have a small integrated GPU on them, just enough to power a screen and set headless mode. And ALL AM5 non-APUs support ECC so far.
A Ryzen 9 7900 or Ryzen 9 9900X would be the sweet spot for power vs performance, but of course you can go lower with a 9600X. At the same time, ECC is only actually important if you run ZFS and a bit flip causes data loss with severe financial consequences. I would prefer a world where everything was ECC, but until we are there, well…
For a home server core with ECC memory support I would recommend an Asus B650 motherboard with a 7900, mostly because the 7900 comes with a pretty good cooler out of the box. We are talking maybe a $800-$1000 investment, to get your power usage down to ~1.5 kWh + whatever your disks draw.
I also recommend OP taking a look at the disks, maybe you are one of those customers that could make use of a 30TB SSD drive. Sure, they are expensive, but take what they cost upfront and will draw 3W-5W at all times, vs what you pay in electricity for four 8TB HDDs. I believe with four 30TB SSDs, you could cut power consumption to below 80W or 2 kWh/day.
Regardless, fewer, larger disks → less power without sacrificing capacity. If you go from 6.5 kWh / day to 2 kWh / day, that would save you a little bit over 1 600 kWh per year. That means $240 saved per year if you pay $0.15 per kWh.
I guess its a feature I am accustomed to with my line of work and it certainly helped me while overseas when TrueNAS decided to take a break.
The server was a decommissioned box from a client site and I decided to use it at home as I had a dire need to run a few VM’s for development plus some issues/needs to address via self hosting. One thing lead to another and now there is someone either streaming a movie or listening to music most of the times.
I would say most times the server would be around 10-15% utilized. Only times it would reach 50% + utilisation is when I am testing something.
I was actually thinking of 9900X with an ASRock AM5D4ID-2T/BCM. The board itself is quite expensive so I was just wondering wheather buy this board or not!
Where I am from a kWh cost about $0.32 on average. So it’s expensive to run.
It is expensive but considering that you’d be saving ~$400 a year in power consumption alone with the figures you are presenting, it’s a no brainer investment for TCO. The board means you will be pretty set for the future, so a $500 board makes sense for you.
Just make sure it has everything you need, at $500 you should not need to do a ton of compromises!
Oooof, yeah, you definitely should consider consolidating to a few small drives then, your best sweet spot options for bulk are basically 3.5" HDDs for 16TB @ $225 or 24TB @ $450-$500.
High capacity SSDs are too expensive right now, 30TB @ ~$3500. We are starting to reach the inflexion point where HDDs will get crushed by SSDs better capacity numbers though, when 1PB SSDs are the norm 30TB SSDs will be cheaper than HDDs, who, at that point, will be at 60TB if they are lucky. I doubt HDDs over 100TB will ever happen, but would love to be proven wrong on that.
there are 8 and 16 core SKUs as well for the price of a Ryzen. Very low idle power (15W)…and they mostly boost to max and run very cold…no fancy cooling required. less power → less heat
I have a link to some very interesting graphs from Phoronix
looks very good. Has power and performance graphs, Phoronix test best test. I get myself a 8224P once I can afford the corresponding U.3 I need for the build. This year was “expensive bike”-year
The power consumption does indeed look good, however, let’s not forget for a home environment the equipment will be idling most of the time. EPYC is built for low per-core watt when in use but draws more idle power when not in use.
This is not a complete Apples-to-Apples as they measure full system draw, but should give a decent idea regardless. Here is the system power for 7900 and 9900X, seems like you want to do some tweaking if going with the 9900X route:
So, at load I think it is safe to say that the 7900 will draw a consistent ~100W in the worst case, you could possibly undervolt this to use slightly less power, while idle seems to be around 20W on both CPUs. The 9900X seems to need a ton of tweaking, the power use is not there yet for me to recommend it over the 7900 for home.
So my recommendation here is 7900 unless you need the 32 cores of the EPYC, but the EPYC is looking interesting too.
it’s not about the cores, it’s about the lanes/slots and memory channels. Ryzen is too constrained and can’t deliver. Power efficiency while sporting multiple NICs, U.3, HBA and possibly GPU is what Siena can deliver. Aside from proper server boards with IPMI and shit.
CPU performance is secondary but you want more cores cause lower clock speed compared to Ryzen.
Thread is about replacing a Xeon Gold 256GB RAM and a bunch of expansion. Ryzen won’t do so your agenda doesn’t make sense.
The graphs are useless in an idle context. because they only look at cpu package power. While the rest of the system like motherboard can use much more power. a lot of 10g adapters will just slurp up 10W idle.
Once you go to a 9000 series cpu it doesn’t really matter idle if it is a 9600 or a 9950X, so that choice should be based on required power. An X cpu can be set to eco mode and be about as efficient as a non -X cpu.
If you want to put in a dedicated gpu, a 10g nic, 256gb ram. then am5 becomes very hard, 4 slots at 48gb is the limit.
Another option is to keep the current computer and use it only for testing. and then to make another computer that acts as a NAS and runs plex etc. 64gb ram is more than enough for those requirements.
That’s what a CPU test is about. Idle power ranges up to 60W on some (modern, not talking about old Xeons) CPUs (especially server CPUs), so this here is a good thing and different. And whether you use the 10G NIC in an EPYC or a Ryzen is the same power, same as everything else. If you put less stuff on your board because Ryzen has little room for expansion, yeah…less idle power, “No IPMI on Ryzen boards = EPYC boards need more power”, surprise surprise.
My personal limit is 80-90W idle and I want a CPU that doesn’t eat up most of the budget just by existing. Siena and Ryzen can do that and that’s what these graphs are very useful for. Not having 60W idle as Intel Scalable is just 50W more for stuff, like 10x SSD or 6x SSD and a 25G NIC.
Sure, the only thing its useful for is comparing platforms. epyc is at the lowest 15w and the intel tested here are 45-50W. But if an epyc motherboard uses 30w more then you are right back where you started.
If comparing desktop platforms an non apu ryzen is the most inefficient cpu idle and this is augmented by motherboard design. For servers it really depends on the motherboard design but clients of server boards are more interested in some load. Do all these server boards support advanced power state management correctly?
If you want power efficient networking, you would probably need to go for 10G SFP+ and DAC cables, or go for 2.5G instead of 10G if RJ45 is a requirement since 10G RJ45 is super power hungry
