Finding the holy grail: low idle power small form factor build with ECC memory and IPMI?

Hey,

I’ve been looking for smaller builds which idle efficiently. Unfortunately, I would also like to have support for ECC memory and IPMI as well, which severely limits the options available. Hence, I’m seeking some inspiration from anyone who’s managed to find this mythical combination at reasonable cost. I can budge on IPMI if needed, since I could use something like a PiKVM.

I have two use cases in mind, which I would use separate builds for:

  1. Backup server that doesn’t do anything else, so should support 3 or 5 HDDs, and have an x8 slot for a 10 or 25GBE NIC. The odd number is for a spare slot for a hot spare, or to burn in a new drive
  2. Proxmox / K8s nodes for virtualization and Docker workloads

ECC eliminates (all?) mini PCs from consideration. They are also too small to fit HDDs.

The only combination I know of with these attributes are the AM5 platform: Ryzen 7000 / 9000 / Epyc 4004 paired with either an Asrock Rack B650D4U or Supermicro H13SAE-MF. This would be great for K8s or Proxmox, but is overkill for a backup target. The Supermicro board alone costs US$434 and I’m concerned with the higher idle power draw of Ryzen desktop processors.

For something less expensive that doesn’t have as much power and idles more efficiently, what combination of processor and motherboard would you suggest?

Things I’ve looked at:

  1. Intel: my understanding is you need workstation motherboards like the W680 for ECC, which are very expensive and even more overkill than the AM5 combination above
  2. Ryzen 8000G: I haven’t seen a successful report of ECC working, even in the pro chips
  3. AM4: if I’m not mistaken, ECC support in the consumer processors only got better in AM5
  4. Ryzen embedded: unobtanium, doesn’t seem to be sold separately to individuals

I guess I was lucky getting my Gigabyte EPYC 3151 board when I got it from an official reseller in my country back in 2023-ish. As I don’t want to throw them under the bus I won’t say which reseller got me one, as they did so as a favour (I’d tried before and was turned down, after reaching out to Gigabyte support they informed/requested said reseller to take my order)

Still not cheap, more then the SM board price you mentioned, but of course that included the onboard 3151 CPU.

So, my suggestion is to find the official reseller for these Gigabyte boards in your country, ask them to sell you one and if they refuse, play it via the Gigabyte customer service route. Just like I did :wink:

Yeah, the situation here isn’t great if you want all those features at a reasonable price. I think your best bet is to get a more consumer level board with ECC support, and then get something like a PiKVM or JetKVM with the ATX control addons and pretend that it’s a BMC. You could also I guess go looking for older enterprise hardware but usually the stuff that’s available for cheap is the opposite of small form factor and has a lot of other disadvantages to boot (high power draw, usually noisy).

Side note about terminology: IPMI - intelligent platform management interface - is a specific software interface you can interact with using things like ipmitool. BMC - baseboard management controller - is the hardware added on to the board which usually provides several interfaces, often including IPMI and a web UI which is what most people want. These KVM options generally do not provide IPMI but do provide the web UI, which is usually good enough in homelab contexts unless you just really like ipmitool.

Apologies if you already know this! Sometimes it’s unclear and I assume I’m providing valuable info when I’m not. :smiley:

7 Likes

How much did you get the motherboard for? I’ve no heard of the Epyc 3000 embedded series till you mentioned it. How is performance and idle power draw? I see that there are a bunch of boards with the processor on it so I’ll check it out.

Thanks a lot for that, I didn’t realize there was a difference. I don’t have anything that has KVM or IPMI, so this is new to me.

It looks like the PiKVM does support ipmitool or Redfish, so technically I could get something that has that support.

What advantages does ipmitool or Redfish support provide, over just the usual KVM access over a browser?

This is the one I got, cost me about 500€ back then.

I haven’t used it much as power here is d@mn expensive so no data on performance nor idle power draw. IPMI and/or BMC will draw at most 10W with the main CPU off, as the 5V standby power is max-ed out at 2Amps.

HTH!

2 Likes

Another option I just found is the Xeon D line from Intel. There are a lot of processors in that series, so lets see if there’s anything promising power consumption and cost wise.

1 Like

What about an Odroid-H4? I believe it allows in-band ECC (IBECC) which uses a portion of the ram to store the ecc data. No IMPI that I can tell but it has very low idle power usage (odroid site reports 2-3W when headless and idle.

I’ve been eyeing them (specifically the plus and ultra versions) for a little while but haven’t bought one yet.

EDIT: ah, just noticed the m.2 is only pcie gen 3 so probably won’t be able to get those high speed NICs you are wanting (at full speed anyway)

1 Like

The price premium on these boards is pretty wild. Even accounting for the higher idle power consumption of Ryzen desktop processors (~20W from the I/O chiplet), that seems to be the best option for now paired with one of the AM5 motherboards I mentioned.

Getting those older Epyc 3000 or Xeon D motherboards are very expensive for much less performance.

i did a breakdown of performance per TDP across enterprise cpus

Epyc on AM5 is the standout if going new and PCIe lane limitations meet your criteria

2 Likes

I would also recommend the Odroid-H4 (Plus) series which would be very good candidates for such applications.

Largely integration with other tooling, CLI, and scripting capabilities. If you want to manage a fleet of machines, being able to programmatically do out of band management operations over an API (or even scripting the ipmitool CLI) is super useful.

As one example, Canonical (the makers of Ubuntu) have a thing called MaaS (metal as a service - sort of a private cloud/cluster management tool) which can use IPMI for power management and provisioning (IPMI from integrated BMCs can usually do things like set BIOS/EFI flags to trigger machines to boot off of the network, for example).

2 Likes

Wow, that sounds really useful. I’ll look out for IPMI support specifically.

That would be a great option if ECC isn’t needed. I’ll definitely keep this in mind and an eye out for future releases from Odroid.

I would say unless you manage a fleet, don’t bother. :smiley: It’s super useful in those cases…but not much otherwise except in specific circumstances where you really like to do things via the CLI instead of web UI.

1 Like

Check this out, It might help!

It’s also based on a “SoC” released in 2008 and Zen 1(!) with no BIOS updates which is likely a big red flag in terms of security. While it does fit the requirements I’d say it’s very questionable whether its worth backpedaling that much. The N97 doesn’t seem much far behind and being much more efficient.

It does support IBECC which is likely as good as it gets unless you’re going to look at industrial boards which are at least 3x more in terms of price.

DFI offers some solutions such as although some specs seems a bit odd…
HPT171|AMD®|Industrial Motherboards|DFI (I don’t think ECC is supported or at least AMD doesn’t list it as a feature)
No IPMI though…

Yeah, at this stage I’ve accepted that the only hope for the situation to improve is for either AMD to fix their issues with the IO chiplet power usage. Or for Intel to recognize how far they’re behind and stop gatekeeping ECC behind the W680.

I’ll investigate the in-band ECC mentioned, this is the first time I’ve heard of it.

I learned about the ASRock IMB-X1231 in this video which ticks almost all of my boxes except IPMI. Available for $330 at MITXPC