AMD Threadripper 3970X under heavy AVX2 load: Defective design? (No, but there is an issue)

[FranzB]

@Lt.Broccoli

Please post your findings in [this thread] at Mersenne Forums, where the author of Prime95, George, hangs out. Prime95 has has had similar bugs in the past, and there’s a good chance you’ve found one.

I doubt this configuration has been tested, and it’s unlikely George has such a high core count system. Those cores are severely memory bandwidth starved for Prime95 for its intended purpose (8 to 12 cores would be a better fit for 4 channels of fast memory for Prime95).

Great idea, will do now!

[jvolkman]

Just to be clear: here are the settings I chose and the sample output. Is this what you’re trying to run?

Seemingly no issues on my 3970x + ASRock TRX40 Creator. This board does have “SRC Spread Spectrum” disabled by default claiming enabling it reduces stability.

[FranzB]

Just to be clear: here are the settings I chose and the sample output. Is this what you’re trying to run?

These are the correct settings indeed.

How long have you been running the test?

This board does have “SRC Spread Spectrum” disabled by default claiming enabling it reduces stability.

I think it’s also disabled by default on my motherboard but I’ll need to check next time I reboot.

[jvolkman]

How long have you been running the test?

jvolkman:

I let it run for > 20 minutes.

[Baughn]

Be aware that some spread spectrum settings can be hidden behind other options. For example, on the Asus motherboard I’m using, I couldn’t disable VRM spread spectrum without first disabling VRM power-saving – but disabling spread-spectrum is what stopped the instability.

[DerAlbi]

Thanks for posting. Please have a look at my thread, i posted an update: the VRM is missing a switching cycle every 4ms actively discharging Vcore so a voltage dip occurs. Oscilloscope screenshot here:
https://forum.level1techs.com/uploads/default/original/4X/e/9/8/e98317514c2e8f565a8c6c76ce078779581b98bb.png
My comments on it in this post:

3970X - Prime95 stability?

Hi, i have not read the post (in detail) yet, but this is getting very interesting. Last night (night for me) I had a session with Wendell who was kind enough to spend time with me probing his board, testing Prime95 and so on. He does not have issues. However i am willing to share another screenshot with you of my oscilloscope findings: [DS1Z_QuickPrint2] What you see is the voltage Vcore-dip on the bottomg. On top i probed 3 of the 16 phases. Only the waveform counts, the ways i probed to phases does not give me absolute values, just an indication of what waveform! THE VRM IS INDEED MISSING A CYCLE. And its worse: it is not only a missing cycle due to a protective feature. I can distiguish this, because the low-side mosfet of the stepdown is turned intentionally ON by the multiphase-controller causing the dip. All phases at once. The CPU does not have a chance. Now, i have had 2 boards with the exact same issue. I have to change the CPU first to really double check that it is not the CPU, but it makes absolutely no sense that the CPU would cause that issue. PMBus-Communication is far to slow to misconfigure the VRM for only one cylce. My current guess is that the XDPE132G5C has some sort of configuration issue (which can be fixed by bios update) or it is actually a hardware fault inside the chips - as if there is a bad badge or something from Infineon. PLEASE tell me that Franz is a German name? Where do you live? Send PM. In your post people suggest it is a Vdrop issue. Trust me, the boards copper layout is solid. Those boards do NOT generate excessive drop with huge variations (meaning the Load line calibration in bios stock settings should be fine). If someone hints at a drop, i honestly think it is the VRM fucking up. We need more oscilloscopes among the people. It is very unfortunate that Gigabyte is not taking me serious. I have no power to get through to someone where a technical discussion about intricate details about the VRM is actually fruitfu…

[terroreek]

Well I think the fix can be done via BIOS update. I am running a 3960x with a Aorus Master. On the F4h bios I could run the test for about 10 minutes before failure. When I upgraded to the F5a BIOS, the test fails immediately with spread spectrum enabled or disabled.

I am running Prime on Linux (Siduction (debian sid)) v29.8. The F4h bios was missing the spread spectrum settings.

[MisteryAngel]

Well i can remember that Gigabyte actually had some,
bug with i believe the F4 bios on the master.
It caused the vrm to run a bit abnormaly warm,
even when it actually has a active cooling fan.
The weird thing is that the Extreme did not seem to have the said issue.
Even though the vrm’s are physically the same.
I believe Gigabyte fixed the problem with the F5 bios on the master.
But did not really state what the actual issue might have been.

But still i kinda doubt that vrm is cullprit when it comes to avx2 heavy load failure.
I kinda wonder if this issue might not be related to the cpu,
actually hitting its powerlimit.

[dprairie_AMD]

Hi,

Sorry you are having issues…can you reach out to me @ [email protected] and we can get you connected with some folks over here to help.

Thanks-
Drew

[oO.o]

Hey everyone, we (mods) are in the process of confirming @dprairie_AMD’s identity. We advise holding off on reaching out to him in the meantime.

Assuming you’re legit, welcome @dprairie_AMD. We’re happy to have an AMD rep among us.

[happyluckbox]

I sure hope that is a real AMD rep! This is a serious issue and I hope it will be fixed in future bios or AGESA updates. My entire workload depends on AVX512, and I just plunked down $5000+ on a 3970x build. Haven’t put it together yet, but I’m dreading having it crash or worse, output incorrect calculations.

[alexdns]

Are you trolling ? Your whole workload depends on AVX512 but you bought a CPU that doesn’t support AVX512 , never did and never advertised it did.

[happyluckbox]

Sorry I meant the smaller AVX, I believe 256?

I’m not really sure about the technical naming here, I just know my software uses AVX.

[FranzB]

I’ve tried disabling Spread Spectrum in my BIOS (it was set to Default, whatever that means) but it didn’t help. Prime95 with min/max FFT sizes set to 16K still fails instantly (within a fraction of a second).

[anon37371794]

Hey everyone, we (mods) are in the process of confirming @dprairie_AMD’s identity.

A quick LinkedIn search reveals that AMD’s director of corporate communications is called Andrew Prairie, usually referred to as Drew.
The mail address in his post also seems pretty legit, seeing as it is listed in several official AMD documents.

[FranzB]

He’s legit. We got in touch with him and AMD is currently trying to repro the problem on their end. Keeping you posted.

[MisteryAngel]

Yes confirmed.

[CybeastRaystriker]

You mean AVX2?

[Mohimm]

If i want to test this under linux (epyc), which program and settings should i use? MPrime? Settings?

[wendell]

I did option 16 then option 2 for most of my testing. On all available threads. Mprime from the cli is fine but I tested windows and Linux