I have a Threadripper Pro 5955WX (16C/32T) processor and am cooling it with an ARCTIC Freezer 4U SP3 air cooler.
When stress testing with Cinebench R23, multi-core test, the CPU gets very hot, above 90C. Are these temps expected for this class of CPU? Or is this cooler wholly inadequate for this CPU? This is my first Threadripper build.
Interestingly, the CPU does not down-clock to cool down. It maintains ~4.5Ghz all-core frequency even at 90C+
I used Noctua NT-H2 thermal paste, and applied it as per recommendation from Noctua for socket wrx80 (12 dots total):
The Arctic cooler seem to get rave reviews around here, and is highly in demand. So I’m trying to understand whether I had my expectations too high for an air cooler, or maybe I’m doing something wrong.
I don’t have a 1-to-1 comparison as I have a 3975WX (32c/64t), but with the same cooler.
If I run mprime on all 32 cores, it processor is dropping frequency because of total package power limit and temps are around 70°C
Tctl: +71.0°C
Tccd1: +66.2°C
Tccd3: +71.5°C
Tccd5: +71.8°C
Tccd7: +69.8°C
If I run mprime on only 8 core and all on the same chiplet, the chiplet also gets to ~90°C hot:
Tctl: +89.8°C
Tccd1: +90.0°C
Tccd3: +49.2°C
Tccd5: +54.5°C
Tccd7: +48.5°C
So it looks “normal” to me.
Myself, I’m happy with the Freezer 4U. In my normal operation I didn’t have any temperature issues yet.
So it looks like you will be able to run it in all core boost mode. You could also limit TDP it in Bios if you are worried.
Benchmarks don’t necessary reflects the daily use case, except you are running 100% of the time whatever cinebench benchmarks tests.
And with his 5955 he’s only got two CCDs, so he’s pumping 300+ watts into a much smaller area. In that situation you’ll run into thermal limits before power limits, whereas with a 5975 and twice the CCDs it’s power limited before thermals.
While >90C is worrying, it’s not over that processor’s 95C tjunction limit that causes throttling. I don’t doubt that an AIO would run it cooler, but you’re also not going to destroy the chip.
It turns out that PBO (Precision Boost Overclock) was set to “Auto” in the BIOS, under the “AMD Overclocking” page. I don’t know why this is set this way from the factory for my ASRock WRX80 Creator R2.0 board. When I set that setting to “Disable” explicitly, the temps are much cooler:
The core voltage under load dropped from 1.244V to 1.219V, which then caused the total core power to drop from 223W to 206W. Finally, the package (total) power dropped from 305W to no more than 287W. All of this caused the max temp not to exceed 86C under Cinebench benchmarking full load.
So I don’t know what “PBO: Auto” setting actually does, but obviously it’s upping the core voltages, and thus the max heat generated. For no real gain in performance, as the all-core turbo still stays at ~4.5Ghz and my Cinebench score is roughly the same too. Darn it, ASRock!
I feel much better at 86C than at 91C. It would be nice to stay under 80C, but I don’t think very many air coolers can do that when dumped with close to 300W of heat.
if you set it to advanced then set the PBO under that menu to disabled and then set the curve optimizer to -20~30 you can get higher freqeuncy per watt
you can also set a custom power draw or have it enabled but set the temp limit lower
you can also increase the turbo ceiling by setting the frequency override to +200
Thanks for the heads up. Negative offset in the curve optimizer seems to be the key for lower temps.
I found this excellent video about overclocking Threadripper 5000 series and will be following along to get a bit more performance, or just to lower temps using the curve optimizer.
I decided to spent the last weekend OC’ing my Threadripper with the Arctic Freezer 4U.
PBO2 settings below:
Stock
% used
PBO2 enabled (mobo)
% used
PBO2 enabled (optimized)
% used
CPU Temp max (seen)
92.8C
95.5C
95.5C
CPU Package (seen)
287W
351.6W
364W
CPU PPT limit
280W
(100%, 280W)
1000W
(34.3%, 343W)
1000W
(35.5%, 355W)
CPU TDC limit
215A
(87.8%, 189A)
280A
(81.2%, 227A)
280A
(84.3%, 236A)
CPU EDC limit
300A
(96.9%, 291A)
350A
(86.2%, 302A)
350A
(94.8%, 332A)
CPU FIT limit (scalar)
“Auto”
10x
CPU VID max (seen)
1.3V
1.306V
1.325V
Curve Optimizer
-15 steps
CPU boost over
+200 MHz
The “% used” column shows how much of the max PPT/TDC/EDC was used. It’s an easy way to see what limit was being hit.
I started with a -30 steps curve optimizer. It was not stable under benchmarking and some of the cores were scratching. Then I spent time adjusting a per-core curve steps, with some at -30 steps and others at -20 steps. This took a lot of time and I thought I had a stable system. Unfortunately it’s hitting something when idling, which causes the system to freeze, requiring a hard reset. So I just decided to stay at -15 steps curve optimizer for all-cores which is stable.
Lessons learned:
At stock, the CPU is PPT limited (uses 100% of the default of 280W).
Setting PBO2 to “Enabled” in UEFI sets the motherboard defaults. It increases the PPT/TDC/EDC limits, and the maybe the scalar to an unknown value. Then the CPU becomes EDC limited (86.2%) but does not hit the max allowed. Heat, at 95.5C is the limiting factor here.
Manually tweaking PBO2 settings allows for the most power to be consumed. EDC once again starts becoming the limit (94.8% used), but does not hit the limit. 95.5C heat is the limiting factor.
Curve optimizer is useful, but is time-consuming. You can go into the rabbit hole here tweaking core-by-core, but it never ends. Some cores that were stable at a particular step, suddenly becomes more (or less) stable, when other core steps are adjusted.
Heat is overall the limiting factor. The ARCTIC Freezer 4U SP3 cannot keep the CPU cool with PBO2 enabled. It’s “OK” at stock speeds, keeping the CPU at 92.8C, which is under the TJmax at 95C. Probably need liquid cooling to dissipate 365W of heat?
Motherboard default “Enabled” setting for PBO2 sucks. It produces heat for no little to no gain in performance. It might even reduce perf! (it’s the reason I created this thread to begin with, PBO2 is “Enabled” from the factory…)
Tweaking PBO2 gives some impressive improvements depending on workload. Double-digits in CInebench, especially 25% faster for single-threaded. Prime95 hits 4.40 Ghz under full load generating heat, vs 4.20Ghz on stock.
The CPU loves to boost, if available thermal headroom (not too hot). For example, in Cinebench multi, a single benchmark run keeps the boost at 4.70Ghz the entire time. But running nonstop benchmarking drops it down to 4.65Ghz. Prime95 (small FFTs) is particularly hard on the CPU for heat. It starts at 4.70Ghz, but steadily drops to ~4.40Ghz after about 30 seconds and maintains it there.
My bus clock (BLCK) seems to be stuck at 98.7 Mhz - 99 Mhz. As I understand, this should be running at 100 Mhz for AMD. Running a slower bus clock is robbing me a bit of performance. For both the CPU (running at 98.7 * 47 = 4.64 Ghz) and the memory (running at 98.7 * 16 = 1579 Mhz)
How do I force it to 100 Mhz? There is a motherboard setting for “PCIe bus clock speed”, which can “Auto” or be set from 100 Mhz - 200 Mhz. But setting it to 100.1 Mhz doesn’t seem to do anything. It won’t let me set it to 100.0 Mhz, 100.1 Mhz is the minimum (which may be a UEFI bug).
you can try raising it a bit to offset “droop”
I tried 100.2 Mhz, 100.3 Mhz. It doesn’t seem to have any effect.
Various utilities (CPU-Z, HWMonitor, HWInfo) all show the 98-99 Mhz bus clock, so it’s unlikely to be reporting error?
there should be a temperature limit thing in the advanced PBO menu
I tried that out and it works wonderfully. I won’t keep it enabled, but will keep it in my toolbox if I run workloads that keep pushing the CPU to TJmax at 95C.
The CPU idle power is high, roughly 70W at idle. Is this normal for Threadripper CPUs? What’s causing the high usage? Most desktop CPUs draw barely a few watts at idle. I believe this does not include power consumed by the WRX80 chipset, but I could be wrong.
Is there a way to bring down the idle power usage?
That’s normal power usage for Epyc and Threadripper, as it’s showing you package power which includes the IO die, memory, and interconnects in addition to the actual processor cores (which you see listed separately).
With my 7713P I see a similar 70 watts of idle power consumption. Because this power is not being used by cores (which can clock down, idle, and park) there is no way to decrease it. Infinity fabric and memory operate at a fixed speed, and this accounts for the vast majority of that 70 watts.
Well that sucks. The desktop Ryzen CPUs are also chiplet design right? They also have an IO die and Infinity Fabric? So why don’t they also have this 70W idle draw?
For a headless server running 24/7 this isn’t great
Maybe I should have gone with EYPC. Even if it has the same 70W CPU idle, there is no additional power used by chipset, since there is none.
because you have a lot more cores and core dies sipping power at once, also a lot more infinity fabric, memory channels and PCI-E lanes, wouldn’t surprise me if your io die would draw more than desktop ones
chipset isn’t considered a part of package power
For comparison I have a 5800x that’s idle package power is 22w. It make sense that yours is higher since you have 4 CCDs plus a significantly larger IO die that has roughly 4 times the IO provisions as desktop zen3.
70W still seems higher than what the package should be capable of. I have an EPYC 7313P that idles at ~33.5W package, and that’s with 4 CCDs instead of the 2 in the 5955WX, connected to what should be the same IOD design. Granted it doesn’t also have an x8 chipset link to continuously maintain.
EPYC has a bunch of settings that affect power consumption, sometimes at the cost of latency for various workloads. If there are equivalents exposed in BIOS for Threadripper, perhaps they’re just set to different defaults?
It’s Zen 3, the memory controllers are on the I/O die. The usual suspects with access to AMD press briefings all list the 5955WX as having two CCDs.
The compute cores wouldn’t be able to consume more than about 4 channels’ worth of memory bandwidth, depending on how things are clocked, but the package as a whole would still have access to 8 channels of capacity. Some Zen 2 EPYCs have similar configurations.
A better way I could have said that would be the GMI links from only 2 CCDs wouldn’t be enough to support 8 memory channel links worth of bandwidth from the IO die.
Ahh you are right, the hwinfo64 screens in the 9th post only show only a CCD3 and a CCD5.
I had wrongly assumed AMD wouldn’t gimp you on memory bandwidth on the lower core count thread ripper pros
