Coming back to document what I’ve tried doing again…
I paid for LatencyMon and got access to the in depth latency tests. This lets me run a Tight Loop Latency Test at HIGH_LEVEL_IRQL which, as I understand, should be testing latency bypassing any particular driver issues by measuring SMIs and CPU stalls.
I continue to see latency peak around 8-900 microseconds, almost a full milisecond of the processor being unresponsive. Here’s a report from a test I just finished: Tight Loop Latency Test HIGH_LEVEL_IRQL - Pastebin.com
After creating a ticket with AMD and providing additional documentation around what I had been testing, they replaced my 3975wx. Spoiler alert, that didn’t fix it.
So to recap, I’ve replaced:
CPU (RMA but same model)
RAM (tried one stick of a completely different vendor from Asus’s qualified list)
Motherboard (RMA but same model)
PSU
Graphics card
Nothing has solved this for me. The latency spikes represent themselves as drops/pops/clicks in audio and the very brief network drop. The test I linked up above I was trying with threading disabled, as well as the HW stream prefetchers disabled which I hadn’t done before.
Still unsure of where to go from here. I would hate to have to ditch this $1k mobo to try something else…
Just wanted to follow up and see if you had made any progress on this as I am seeing similar issues on my end but with random system hangs.
I have roughly the same setup as you (CPU & MB) but run a 6950XT for the GPU. For whatever reason, I seem to have random system hangs where I will open up file explorer (any program really) and will get a spinning wheel where nothing happens then a “not responding” error and all of the sudden, everything will snap back into place. It is nothing I can recreate, and it happens at random. I’ve also run latencymon and see similar latency spikes as well - even with a fresh install of Windows 11 for Workstations.
Much like you, I am at a loss here but looking at your last post, you have pretty much replaced everything except the OS drive. Have you tried replacing that to see if you still get those latency spikes? I know this is just a shot in the dark but I would be curious to see what the results would be.
Sorry I didn’t see this earlier. Yes I replaced the OS drive. I actually originally thought this issue was due to me running 2 x 1TB 980 pros in raid 0, so I swapped to a 2TB 980 pro, no raid. Didn’t effect it at all.
I actually moved away from looking at this as the main issue and focused on other stuff. Like the network drop outs. I dropped the speed down from 10 gig to 2.5 Gbps and had far better luck. I then got a new SFP+ adapter for the switch side of the connection and currently run at 5 Gbps reliably. I don’t know whether the Cat 6 cable is too long, the SFP+ adapter isn’t compatible with the switch or x550, or something else. I’m fine running at 5 Gbps so will just continue with this. I’ve tried running with the network adapter disabled in the Bios, cable unplugged, and still saw latency spikes.
The audio clips, dropouts, pops I was experiencing may have been entirely due to a faulty GoXLR. I’ve tested a brand new one (same mic, cable, power, etc) and have had no issues.
I’m running with threading enabled, but any setting in the Bios that would allow the CPU to conserve power turned off (I can get a list together if you want). These changes have seemingly eliminated the feeling of a stuck/hung system as you’re experiencing.
I’ve done a fuck load of reading about issues like this and it might be power related. If there’s noise coming through the line or very quick voltage drops/spikes, we could see latency spikes like this. But I’m running this off of a Cyberpower PR2200LCDSL and a Furman Powerstation-8 conditioner. I’ve worked with my utility company with monitors placed both at my meter and on the same internal circuit that the PC is plugged into… they don’t see anything wrong.
I just ran latencymon’s HIGH LEVEL IRQL test again overnight and am still seeing spikes. They are incredibly infrequent now (I saw it at 12 hours with no spikes).
If you’re also dealing with this, I would try:
Run at 5 Gbps or lower
Only use one of the network adapters
Turn off anything you’re not using in the Bios
Make sure whatever power mode you’re using in Windows has the processor minimum state set to 100% and all other power savings disabled
Go get the actual latest chipset driver direct from AMD (The Asus page for this mobo does not have the latest)
Go get the actual latest ethernet driver from Intel for the x550 (Again, Asus page is behind)
Ensure you’re running the GPU boost locked (don’t know what this would be called for team red) during the duration of latency spike testing
Don’t do any crazy registry stuff, that road is painful and will probably result in you reinstalling Windows to get back to a clean slate
As you can see from this thread, I’ve dealt with this for months and I don’t know what else to do or what else to tell someone else dealing with this. I’d be very curious to see your latencymon test results and what all you are running hardware wise (including what’s plugged into it).
The latest BIOS and BMC firmware + limiting the PCI slots to PCI 3 has resolved seemingly all of my issues. Note that I’ve left my BIOS settings to disable C states and any power savings as well.
I have not run more IRQ tests because as long as the interrupts aren’t noticeable to me in any perceivable way, I don’t care much about what the numbers show.
Just wanted to chime in here and thank you for documenting all of this. I’m also on a 3975 and the same mobo and I know EXACTLY what stutters you’re talking about. Its extremely hard to find more info online about it. Audio dropouts, keyboard lag and keystroke repeats, mouse cursor hanging and then catching up. All of it.
My issues can be repeated whenever my system is under heavy load. Say when I’m rendering in Octane, or when After Effects is caching frames.
I thought I had the latest BIOS but will have to double check that I wasn’t getting it from the ASUS site like you mentioned. I also saw mention somewhere that disabling BMC would help but it hasn’t. Also saw how disabling SMT might help, and while it did A BIT, it didn’t fix it completely, and I benefit from those threads so I’d prefer that to be the fix anyway.
Disabling PCIE4 also wouldn’t be ideal as I have a multi GPU setup for rendering and I pass a ton of data over PCIE.
So all that said, do you think we have similar issues? Any advice 6 months later? Any help at all would be super appreciated. I have parts for an i9 14900k build on the way, and I’m in the “last ditch effort” phase of finding out whats going on here. Otherwise I’m jumping off the AMD ship and going back to Intel.
I completely understand your frustrations. It kills me that I have this beast of a PC and it can’t do basic PC… stuff.
This is not fully resolved for me, and I’ve given up trying. When I tried to be a good end user and report my issues to Asus support, I was almost essentially laughed at by their support. I understand why, but it still sucked. It’s also incredibly difficult to try and explain to their phone support what I’m going through (no disrespect to phone support peeps, it’s just this issue is complex). I was told that there are thousands of happy customers and they’ve never heard of this issue. While I believe that’s true, I also believe it’s true that most people wouldn’t actually notice.
I also have no idea if it’s an Asus or AMD issue. I have replaced every component of my system, including the processor. I have had issues across Windows 10 Pro, 10 Pro WS, and 11 Pro. I have not had the opportunity to try a different motherboard make/model, just a replacement of the same.
Best I can tell you is to install latest BIOS from Asus, install latest chipset driver from AMD, disable what you don’t use in the BIOS.
I would love to upgrade to the new 7000 pro series, but I cannot justify it until I see DPC latency numbers from someone. It’s just too much money. I’m somebody who needs PCIe slots + lanes, so the Pro platform is right for me.
unfortunately, I only found this thread after searching for exactly what my problem with my new built machine is.
Setup
MB: Asus PRO WS WRX80E-SAGE SE WIFI II
CPU: AMD Ryzen Threadripper PRO 5975WX
RAM: 8x64GB DDR4-3200 Samsung RDIMM reg ECC (M393A8G40AB2-CWE)
GPU: 24GB MSI GeForce RTX 3090 GAMING X TRIO
OS: Win10 Pro (but I tried all others without luck)
I tried everything mentioned in this thread without success (besides any swapping of hardware). By accident I found a solution for my case - but based on the fact that @TheLegend_TubaGuy also tried a naked system, it is probably not a solution for him.
I’m also running a M.2 Card that came with the MB with 4 NVMes. When I switch the setting for it to on (so 4x/4x/4x/4x mode, Asus calles it Raid Mode), the Card works as intended, but my LatencyMon is more over 1000μs then under and I get nasty halts.
As soon as I disable the raid mode, the card does not show the discs in windows any more (as intended), but my latency is down to 90-130μs with occasional spikes to 476μs (I think caused by my GPU clock changing, didn’t investigate further because a 500μs is still not noticeable).
At this point I stopped for 2-3 month trying to fix it because I had to use the system for work. Maybe that helps anybody, or maybe someone has an idea how to fix my case?