Two weeks into this and I’m out of ideas. Posting in case anyone has seen this failure mode on Turin, or on this board specifically.
Symptom
Under memory-latency-bound load at high thread counts, the machine dies instantly. Not a panic, not a reboot — it goes completely dark and will not POST again until AC is physically removed (PSU switch off or unplug). The power button does nothing until then.
Originally found by running Agisoft Metashape, which killed it unattended. Minimal reproducer:
stress-ng --stream 128 --timeout 300s
Depending on configuration this kills it anywhere from a few seconds to a few minutes. It is probabilistic — some configurations survive for hours and then die, which made this miserable to bisect.
The part I can’t get past
There is no error record. Anywhere. Every reporting path this platform has comes back empty:
-
No kernel panic, no oops, no MCE
-
rasdaemon/ EDAC: zero CE, zero UE on every channel, with ECC confirmed active -
I additionally enabled DRAM Write CRC, Read CRC and RCD Parity in BIOS — still zero
-
BMC SEL: nothing at the time of the crash (only “Power off/down — Deasserted” when power returns)
-
No BERT/ERST record on the following boot
-
netconsole to another box on the LAN captures nothing — verified working with a test message first, so the pipe is good. The kernel is executing normally and then the lights go out.
That reads like a hardware protection latch or fabric halt below the OS, but nothing will tell me what tripped.
Hardware
| CPU | AMD EPYC 9575F (64C/128T, 400W default TDP) |
| Board | ASRock Rack BERGAMOD8-2L2T, BIOS 10.09, BMC 10.03.00 |
| RAM | 8× G.Skill F5-6000R3036G48GE8-G5N (48GB RDIMM, 384GB total) |
| At SPD default 4800 MT/s, 1.10V — not EXPO, not overclocked | |
| PSU | Seasonic PRIME PX-2200 |
| Cooling | 360mm AIO |
| Storage | Samsung 990 Pro 2TB NVMe |
| GPU | RTX 5090 (headless box, llvmpipe desktop) |
| OS | Fedora 44 (7.1.7 + 7.1.5), Ubuntu 24.04.4 (6.8), Ubuntu 26.04 (7.0.0-14) |
Board has 8 DIMM slots, so 8 of the CPU’s 12 channels are populated. NPS1.
Ruled out
Thermal — 57–60°C under the failing load. (There are CPU_THERMTRIP events in the SEL from the very first boot — that was me forgetting to plug in the AIO fans. Nothing since.)
PSU — 2200W unit, package draws 320–400W. Not close.
GPU — removed entirely, still crashes. There were PCI PERR bursts in the SEL early on; those stopped after a BIOS update plus moving the card to a different slot, and the crashes continued unchanged. Red herring.
Storage — NVMe SMART clean, 0 media errors, btrfs scrub clean.
Memory — this is the one I was most confident about and it’s dead. It crashes with a single DIMM at 3600 MT/s, and passes 900s with all 8 at 4800. Tested 8/4/2/1 modules, multiple different modules, multiple slots. Module count, speed and channel population are all irrelevant. Combined with the zero ECC/CRC counters, the DIMMs are as exonerated as I can make them.
Distro / kernel — fails on Fedora 7.1.7, Ubuntu 6.8 and Ubuntu 7.0.0. The rate varies a lot (6.8 is much more tolerant, 7.1.7 dies in seconds) but nothing is immune.
Power management — pcie_aspm=off, amd_pstate=disable, processor.max_cstate=1, idle=poll, nosmt, and a 320W package cap via HSMP each reduced the failure rate and none of them fixed it. Every one of them looked like the answer for a while, then failed on a longer run.
The discriminator: load type, not power
stress-ng --cpu 128 runs at a flat 400W indefinitely with no problem.
stress-ng --stream 128 kills it at ~320W.
So it isn’t total power. It’s something about saturating the memory path. DDR bandwidth sits at ~255 GB/s (83% of the 307 GB/s the SMU reports as max) during the failing load.
Thread count is the strongest single correlate. Anything running ≤64 active threads has survived 900s runs repeatedly, including all 8 CCDs at one thread per core. 96–128 threads fails.
A secondary observation I can’t fully explain
Using taskset to pin work to specific CCDs (8 cores + SMT siblings each, per shared_cpu_list), at constant 112 threads and constant package power, the failure rate depends on which CCDs are loaded:
-
All CCDs except CCD4 → instant fail, 3/3 attempts
-
All CCDs except CCD3 → pass
-
All CCDs except CCD7 → passed many times; eventually failed once at 300s
I built a whole model on this and then it broke, so I’m presenting it as a rate difference across a limited number of trials, not a proven mechanism. But it’s suggestive: the motherboard has no knowledge of which CCDs are active and supplies identical rails either way, so a VRM/PSU/memory explanation struggles with two runs at identical load and opposite outcomes.
Running with CCD7 isolated (isolcpus=56-63,120-127, 112 threads) has given me 6 days uptime and several completed Metashape jobs, which is by far the best I’ve had. Not a fix, but livable.
Two firmware bugs found along the way
Independent of the crash, flagging for anyone else on this board:
-
BIOS TDP/PPT does not enforce. Set TDP Control and PPT Control to Manual, value 320 — the menu displays 320,
turbostatmeasures 379W under load. Setting the identical limit via HSMP (/sys/devices/platform/AMDI0097:00/hwmon/hwmon*/power1_cap) works correctly: 319.7W measured, andpwr_current_active_freq_limit_sourcereportsPPT Limit. So the SMU is fine and the BIOS path is broken. (It also only took effect at all after a full AC power cycle — a warm reboot silently discarded it.) -
Platform First Error Handling = Disabled has no effect.
GHES: APEI firmware first mode is enabled by APEI bitstill appears at boot and_OSCreportsplatform retains control of PCIe features (AE_SUPPORT). Presumably a large part of why nothing ever gets logged.
The thing that’s nagging me
ASRock’s release notes for BIOS 10.02 and 10.09 both state: “Only supports Turin CPU below 400W” and “Only supports DRAM frequency max. up to 4800MHz”. The 9575F is a 400W default TDP part. Their CPU support list shows the 9575F as supported from 10.02 onward.
Also — the BERGAMOD8-2L2T manual (v1.0, July 2023) lists support for EPYC 9004 and 97x4, i.e. Genoa and Bergamo. Turin support came later via BIOS. ASRock has since released the TURIND8-2L2T, which STH commenters note appears externally identical to this board, with speculation that the changes were for memory signal integrity.
Questions
-
Is anyone running a 400W Turin SKU on a BERGAMOD8-2L2T successfully under sustained memory-bound load?
-
Does anyone know what actually changed between the BERGAMOD8-2L2T and the TURIND8-2L2T?
-
Given PFEH won’t disable and firmware-first won’t relinquish, is there any way to make this platform say what tripped? I’ve tried
pcie_ports=nativeandhest_disable— the kernel takes over MCA decoding and still sees nothing. -
Does “fails at 96+ threads, stable at ≤64, no error records at any layer, memory eliminated” read as CPU or board to you? I can’t swap either part to find out.
Happy to run anything useful. Thanks for reading.