Epyc 9005 (Turin) motherboard options?

I am curious about the 9005 series (considering 9115).
I was wondering if anyone has experience with any motherboards currently on the market:
H13SSL-NT
H13SSL-N
GENOAD8UD-2T/X550
S8050GM2NE
or any other.

  • Do all of these support 9005 with a simple bios update? Supermicro states 9005 support requires revision 2. How does one find it? I see no revision info listed on retailers’ websites.
  • Good IOMMU groups for virtualization?
  • How big of a deal is 12 RAM channels n Supermicro vs 8 with the rest? (at the cost of worse PCIe expandability)
  • generally, how happy are you with them if you have any of these?
  • which are pros and cons worse mentioning in your opinion?

Thanks

2 Likes

ALL Genoa and later EPYC require 1 DIMM per channel (EPYC 9004 and 9005 are 12 channels) for optimal performance
any more or less results in up to 30% loss in benchmarks according to my testing.

That limits you to 192 GB loadout minimum and multiple up to 1.5TB)
I’ve built on the H13SSL-N, H13SSL-NT, and MZ33-AR1

Gigabyte is better for compute, storage applications, and huge RAM loadouts

Supermicro is better for GPU acceleration (clearance issue on gigabyte boards)

all of them can be upgraded through the IPMI with an incompatible or vendor locked CPU installed.

rev 2 is labeled on the board when they post it on eBay
I haven’t seen a sub rev 2 H13 board in the wild

3 Likes

Thank you for the informative response.

I was eyeing a new board, and NewEgg does not seem to post the revision.

What is your opinion on Asrock and Tyan boards for 9005 (like the GENOAD8UD-2T/X550 and S8050GM2NE)? Do you have experience with virtualization for any of them?

I’d like to run TrueNAS in a VM.

Thanks!

Also… Do you know any silent/quiet CPU cooler options for 9005 (sp5)?

I have deployed more virtualization hosts on EPYC than anything else, but have also built a few EPYC TrueNAS deployments. There is no better host on the market.

Have not used either with SP5, have an ASRock Rack board for an EPYC on AM5 build but my RAM got lost by USPS.

@JayVenturi posted the best resource for cooling EPYC sp5 socket on the internet, he just graced us by posting it on this forum

2 Likes

Thank you

What is your opinion: Is the extra perf of 12 channels worth it with Supermicro MBD-H13SSL-NT considering its drawbacks:

  1. Higher cost to populate all the slots with RAM sticks (if I were to take advantage of the 12 channels)
  2. No built-in Intel 10Gbe (I’ve heard bad things about the Broadcom chipsets on TrueNAS forums)
  3. Only 3x full-sized PCIe slots
  4. RAM QVL only has 1 model (???)

VS the other boards with more full sized PCIe or Intel networking?

I am planning a 10Gbe (would love 25-50 Gbe, but there are no reasonable options to connect to my laptops) home server with multiple VMs, self-hosted services, home automation, data backups, and possibly camera footage storage (no cameras yet, but definitely on the planned list).

Thank you!

absolutely
get 16 gig dimms and enjoy

there’s the H13SSL-NT, but uses broadcom

pretty rare to need more now adays

we prefer SFP cards for flexibility

There are equivalents available from other manufacturers, but we used models identical to the approved ones from the QVL available direct from SK Hynix but substantially cheaper. Don’t have the part number handy and haven’t built a Turin yet, only a bunch of Genoa generations.

All the computers at my home are laptops, and a thunderbolt 25Gbe do exist, but they are huge and cost $900 lol

there’s the H13SSL-NT, but uses broadcom

I hear everyone say that Broadcom-based networking is very unreliable. I see everyone recommend Intel-based NICs for 10+ Gbe.

If the only difference of the cheaper H13SSL-N the lack of Broadcom 10Gbe, then getting it and pairing with a PCIe SFP card seems like a good choice to me.

pretty rare to need more now adays

HBA + Hyper M.2 card (for Optane SSDs) + NIC is already 3 slots.

And then you may want a GPU (or even 2 - one for compressing camera streams and one for transcoding Plex/Jellyfin) , a second HBA eventually, etc., etc.

I am willing to invest into a proper enterprise CPU + MoBo for the PCIe lanes. If the slots are limited, then this defeats the purpose for me personally. That’s how I look at it. But you are totally correct: most people don’t need more than that nowadays! :slight_smile:

There are equivalents available from other manufacturers, but we used models identical to the approved ones from the QVL available direct from SK Hynix but substantially cheaper. Don’t have the part number handy and haven’t built a Turin yet, only a bunch of Genoa generations.

Did you have any issues? Would you be able to look up the part number by any chance? I am scared of getting incompatible sticks and having to deal with the pain of return process. I read horror stories where people spent tons of $$$ on RAM that was supposed to match (speed, latency, etc.) and yet it didn’t work. MoBos nowadays are too finicky!

Also… Do you remember how much you paid? The DDR5 RDIMMs costs that I looked at are insane.

THANKS!

Looks like H13SSL-NT/H13SSL-N is a bad choice for a quiet build: the board does NOT like low RPM fans - it panics when a fan is below a (high) threshold and compensates by blasting them full force:

https://www.reddit.com/r/supermicro/comments/1fhr9o7/x13saef_fan_ramping_up_and_down/

“Only jet engines need to apply”

what NIC are you tryin to run?
dual 10 and 25 gig cards since Connectx-4 days have been x8

quad 10 gig and my dual 100 gig cards are x16

if you’re doin this and don’t have the flash storage, I’d strongly suggest U.2 over M.2
random iops, price per TB, heat dissipation, and drive endurance are off the charts
brand new 4TB Intel U.2’s are $250 all day on eBay

Use Phantek T30’s or Noctua and set to max
really not loud at all

U.2 / U.3 nvme express is probably the best and I echo with TryTwiceMedia

Phantek T30 using the little black switch (next to fan motor) can scale from 300rpm to 3000rpm, in BMC set fan minimum to 450rpm if the 300 rpm gives alarm, or set fan curve to never be below 450 rpm, or set fan alarm to be 150rpm - any of these methods would solve the low rpm warning.

Also even at 3055rpm (as reported at max on my bmc for the fan) the T-30 is relatively quiet if one can say “quiet” in the same sentence as “3000 rpm”

on amazon the micron 3.8TB u.2 are less than $450, the 7.8GB are $698

how can we help you?

1 Like

Apparently (from researching), not all SM mobos allow to change this, and I am not clear if this one does.

U.2 / U.3 nvme express is probably the best and I echo with TryTwiceMedia

I already got 4x 118GB Optane NVMEs when they were on sale on NewEgg. I was planning to use them for metadata, but I am getting a feeling now they may end up being too small for my pool size (6-9x 14TB drives in raidz2 probably, so 64-98 TB usable space.

2x 118GB of Optane (+ 2x for redundancy) probably won’t be enough if we are to follow the 0.3% rule.

Not sure if I should sell them off on eBay now or keep them for something else. They are rare/discontinued now. Shame that Intel abandoned this cool tech.

Optane and VROC worked well and it is a shame Intel half-assed the implementation.

Some issue were easily avoidable, like selling motherboards for $1200 but making the user go buy a separate $210 motherboard plug in key to enable the VROC

Other BS took place like limiting optane products, distribution channels, and charging $ for every step along the way.

Hence my trepidation on optane, it was cool, it worked, but quite a complex house of cards to fully impliment

1 Like

We only use M.2 as boot drives in enterprise deployments

a pair would make a fantastic boot solution

Ironically, their flagship and most hardware restricted product: persistent memory was the easiest to configure and use but was only validated with a few high end CPU’s and available through OEM channels in complete systems.

Intel is makin the same mistake with their Flex GPU’s now

Was it a B2B purchase? Because I am not seeing a way to buy RDIMMs directly from Hynix as a private party.

Hynix stuff through retailers like Newegg is not cheap: Hynix DDR5 that I saw is ~$220+ per 32GB stick. That’s roughly $1,000 per 128GB just for RAM.

Any way to get good RAM for cheaper than that?

We do not have a B2B relationship with Hynix, fleeBay is your friend here

It’s $900 for the RAM we bought (from memory last year), but that was 4800 MT/s 192 GB’s.

Remember that EPYC Genoa and Turin requires all memory channels be populated for full performance. So you’re dealin with 192 GB minimum loadout, 384 GB for a standard loadout and more for more.

I haven’t chased faster as I haven’t built a Turin yet. This year, we do not have any large deployments requiring Turin so unsure how this will work out.

Even on fleebay it comes out to ~$1000

Do you think DDR5 rdimms will drop in price significantly any time soon?

always
new pulls are constantly in a race to the bottom as bigger and badder are released
It’s typically enough price delta to purchase aftermarket rather than with server from OEM that admins replace RAM on arrival.

1 Like

Well, it seems they haven’t come down much since your purchase last year because $900 still seems to be the cheapest, and most postings are even higher.

It’s insane how much more expensive it is compared to DDR4. Almost double.

1 Like