AMD Epyc Milan Workstation Questions

Look for or ask Asrock Rack for an updated BIOS or beta BIOS to fix the missing inventory.

I have found low profile 6" USB header extenders to solve the issue of the large molded USB cables from cases not fitting. Allows you to move the male-female connection off the motherboard and to the side or under the motherboard or hidden elsewhere.

I buy them from moddiy.com. I have two more ordered but in transit for the past month because of the Chinese lockdown.

I’ve used them on just about every motherboard I own.

2 Likes

I’m unsure about the terminology - is FCLK the same as Infinity Fabric Frequency in the context of AMD zen3? I would doubt that changing memory clocks changes IF freq, given that it is possible to run any Rome or Milan CPU with any of DDR-2666/2933/3200. And this comes with a warning that choosing the “wrong” RAM clock will mismatch with (not affect) the IF freq.

Here are the actual options I get, with sensibly smaller steps above 1600MHz:
image

(I have no idea why 400MHz is out of the numerical ordering)

I would have no problem lowering the RAM frequency of my 1600MHz DDR4 to 1333MHz for testing, although then I don’t know how I should assess the result w.r.t. FCLK/IF?

Thanks! I had hoped that exactly this would exist. Might try it - although so far I have not found a good uATX chassis that is otherwise suitable for this machine, so I might end up with a “compact” ATX chassis in the end.

I run my 7402P which has a AMD specified max of 2933Mhz memory at the the stock XMP of 3200Mhz and just ignore that the memory is running asynchronous with the cpu.

I have detected no performance degradation issues even though AMD warns there might be on certain applications.

It is just the recommendation in their tuning guides to not do so.

I tried once to overclock the memory by selecting a higher memory clock in the BIOS but the host would never boot. So just fell back to the 3200 XMP settings.

That was my experience with Rome too, at least using Intel mlc I could not find any significant (theoretical) performance drawbacks with using 3200MT/s memory clock. On most metrics the higher frequency looked better. All this was from synthetic benchmarks, I have no real-world test case where I expect memory speed to matter much.

Has your ASRock Rack board BIOS got the same “overclock” options as I found? Did you try to change memory timings in addition to frequency?

Yes, same experience with Intel MLC. Plus I have seen no differences in my scientific applications that are sensitive to memory clock.

I did play around with all the memory settings that the BIOS offered me. First just the simple clock frequency change to see if that would work. Wouldn’t boot when I tried to increase but booted when lowered.

Then I went after memory timings as that has been my goto choice for application speedup on Ryzen. Tried to tighten up the primary timings like latency and such with some hints from Ryzen Timing Calculator and even tried some memory timings that I had used on TR2920X and my X399 board. But every time I tweaked any individual memory timing, it wouldn’t boot. Wasted half a day trying everything I could think of and then just gave up and accepted XMP 3200 stock timings and moved on.

Wish I could have gotten at least one thing changed so I could have benchmarked an application or MLC run to see if the effort would have been worth it. No luck.

1 Like

The drawback with changing a lot of variables at the same time (mainboard, CPU, and cooler) is that it becomes hard to monitor the results of the changes. Now, with my single-fan Arctic Freezer TR4-SP3 (as I’m waiting for the replacement for the fan that was DOA), I’m unsure what to expect in terms of temperatures.

When loading the 7443p to 200w it reaches around 69 degrees celsius, and the Freezer’s (temporarily) single fan runs at 1500-1600rpm. I have no idea whether this is normal.

$ sensors
k10temp-pci-00c3
Adapter: PCI adapter
Tctl:         +69.5°C  
Tdie:         +69.5°C  
Tccd1:        +65.0°C  
Tccd3:        +69.5°C  
Tccd5:        +65.8°C  
Tccd7:        +68.0°C  

$ sudo ipmitool sensor reading FAN2_1 PWR_CPU
FAN2_1           | 1600 # <-- CPU fan
PWR_CPU          | 200.000

What do you who own this or similar chips say, are my fan thresholds reasonable? I don’t know to what extent the ASRock BMC needs tweaking. I have not found any sure guidelines about what temperature range Milan should work within.

Edit: Max fan speed for the Arctic Freezer TR4-SP3 is specified as 2300rpm.

I too intend to try changing at least something in the memory timings, to see if it is at all possible. I hope to get around to it later this weekend.

I saw reported max temps with a Dynatron A26 in the high 50’s.
I think your single fan is hurting your reported temp.
But I can’t find anywhere any spec for what the max temp for Epyc is.
Always reported as ???

1 Like

Thanks for your input! High 50s sounds more reasonable, I’d like to aim for that. Maybe the missing fan plays a role here. When writing the post above I reflected on whether the missing fan could be the problem, and I kind of landed in that it should not, because

  1. the single fan still doesn’t reach its max rpm - shouldn’t BMC simply compensate for a missing fan by raising the fan speeds?
  2. As the fans sit in a pure push-pull config, 2 vs. 1 should affect pressure, but not airflow so much, and the heat transport capacity scales with airflow

Although I might be totally wrong about (2).

Re (1), now I’m simply assuming that BMC lets all fans scale with the CPU temp, at least when that temp runs away.

Now I tried setting all fans to 100% manually via IPMI, then the single Arctic fan reaches 2000rpm (not 2300), and CPU temp stabilizes at 60c. My ambient is a bit lower than yesterday’s measurement though, this morning before changing the fan settings I got 65-66c at 200w/1500rpm (vs 69-70c yesterday). So I think I need to configure the fan settings - I’m just surprised by needing this as I thought the defaults would be fairly allround.

It’s never 2X fans = 2X greater cooling. More like somewhere around 125% better depending on the fin stack restriction. High FPI equals greater restriction that requires more static pressure and higher revving fans.

Lesser FPI equals less restriction with slow moving fans that have more airflow produce better temp results.

Usually the recipe is a low FPI with a push-pull fan arrangement with high airflow fans from lower speed fans drop temperatures the most.

1 Like

I made a little discovery: ASrock rack have CAD files of some of their boards on their website, including the ROMED8-2T

I was thinking about 3D-printing a fan-holder/guide to cool the VRMs on my ROMED8-2T, rather than some the janky velcro I currently have, and come across these.

You just need a STEP file viewer, e.g. the free NIST STEP File Analyzer and Viewer - which is quite a nice one as it outputs files for a web-based viewer:

7 Likes

Hello guys,anyone here use Gigabyte MZ31-AR0?

1 Like

Since there are so many ROMED8-2T owners around here: Any idea how many amps I can draw from an (each) individual fan header? I am heaving clearance issues with a PCI card (technically, I am playing Tetris with a few). Consolidating some chassis fans onto one header (with Y splitters) would geometrically solve the problem …

I’ve never seen the spec on the fan header amperage limts anywhere.
Since the fan headers use the the proprietary 6 pin fan headers for the server 2U-4U server fans, I bet it can handle a lot considering someone might fit a Dynatron or Delta fan cooler.

A standard motherboard fan header can supply 1A @ 12V. Lots of other headers are rated for 2A or even 3A for water cooling or AIO coolers.

So I don’t think that putting a couple of standard PC fans on a fan splitter is going to cause any issues.

Just for future reference, 4 fans on one header, staying well below 1A in total, is not an issue at all. Tried it, it worked. Thanks @KeithMyers

@s-m-e
We folks in the x570D4U-2L2T thread have confirmation from Asrock Rack support that those 6-pin headers are rated for 4A. I’m not sure if they’re all the same across all their boards though and SP3 boards might have even more juice on the headers.

4A is plenty unless you’re going full jet engine with 15k rpm fans.

2 Likes

Latest iteration of 7443P build, this time using the Arctic Freezer 4U SP3 cooler.

The fans it came with are dogsh!t, they went straight in the bin. I strapped 2x Noctua Industrial 3000rpm to it instead, running at 1800rpm.

Runs at ~51degC with all 48 threads maxed out running Prime95, 3100Mhz all core boost, ~205W package power.

This is in contrast to the ~72degC my previous builds using Noctua SP3 92mm coolers. Incredible difference.


8 Likes

Thanks for the review of the Arctic Freezer 4U solution. That is a big difference over the Noctua solution. Much more than just more heatpipes (6 vs. 8). More finstack surface area and the great fans probably the greatest difference.

Such a pretty build! Still debating on going for 2nd gen Rome now or waiting for intels new HEDT release or whatever it is… although I’m tempted to bite the bullet now and upgrade and not worry about new products

They’ll always be something faster coming along real-soon-now. Whether or not it’ll actually be in stock is a whole other matter these days…

And has anyone price up DDR-5 RDIMMS? I suspect the pricing will be rather unpleasant for quite some time…

For you folks with onboard 10G-NICs, what sort of temps are you seeing in operation?

In a 26C room, my CPU is hovering at 29-30C idle, while the NIC is sitting at 71C on my H12-SSL-NT.