Asus Pro WS WRX80E + 3975WX Interrupt Latency

Hey everyone, long time lurker first time poster. I have an Asus Pro WS WRX80E-SAFE SE WIFI paired with a 3975WX, 256 Gigs of Kingston ECC (8x32) RAM running at 3200 MHz. I have an EVGA 3090ti FTW3 Ultra GPU and a Corsair AX1600i power supply. For my drive, I’m using a single 2 TB Samsung 980 Pro in the first m.2 slot. Drivers and bios are the latest from both the Asus website and the AMD website (for more up to date chipset drivers). I’m running in the High Performance power plan and minimum processor state is set to 100%.

I have been struggling to get a stutter free system. I had hoped I was seeing the effects of the fTPM bug, but now with a discrete TPM installed, I’m still seeing stutters. I’ve configured in the Bios to use the discrete TPM and have checked this via Windows. I recently reinstalled Windows from scratch to eliminate as much as I could, but I’m still having this happen. I’m unable to play a song in Spotify start to finish without some sort of audio dropout. It’s like it misses a beat or something. This performance loss seems to be present in other applications but crops up in really strange ways. Playing games is extremely difficult with mouse input sometimes lagging behind what I’ve actually clicked.

I downloaded LatencyMon and have been using that to check what’s going on. I have disconnected everything from my computer and removed all other PCIe cards (leaving just the GPU) and still see ntoskrnl.exe spike to well over 2 ms which causes LatencyMon to report that my system appears to be having trouble handling real-time audio and other tasks.

Is there something else I’m missing here in terms of performance? I work with large scale architectures for my day job but cannot for the life of me pin down these random performance blips I’m seeing in Windows.

I welcome any input! I’ve tried to include as much detail as possible but if I’ve missed something, please let me know.

3 Likes

If the GPU set to high performance all the time?

I get the issue whether or not I have boost lock enabled. Not sure what else you’d mean by GPU being set to high performance.

Yup, all power connections made. I’ll try running memory a bit slower.

1 Like

I’ve tried running memory at 2666 and back at 3200, I get no difference in the highest latency reported in LatencyMon. I’ve disabled C states in the bios, this seems to have helped lower my average latency, but I still get spikes. I also set BME DMA mitigation to enabled, this seems to have helped with the highest latency I’m seeing. I’ve tried running in Windows 10 “clean boot” and still saw latency spikes. I put my other PCIe cards back in and plugged everything back in, this does not seem to have made anything worse. I’ve tried it with and without wifi/bluetooth enabled, doesn’t seem to effect it.

The only thing that I haven’t really messed with is swapping out the GPU at this point. I have a spare 3080ti I could try, but LatencyMon usually reports ntoskrnl.exe as the highest reported DPC routine most of the time, not the Nvidia driver. The LatencyMon FAQ suggests trying to turn off the pagefile but that seems extreme. I’ve disabled the search indexing service, I’ve disabled windows store auto update checks, I’ve disabled windows security periodic scans, still get spikes.

TLDR: I feel like there’s something bios and/or windows settings related that is causing these performance dips I’m experiencing. I’m just not sure what else to try and tune.

I saw someone else with issues with that board before, I have a feeling it might be the newest bios causing them. Maybe going back a bios version will help?

1 Like

Anything in Event Viewer at the time the songs blip?

If you fire up a USB Live Linux distro do you have any issues? Will at least tell you if it is a Windows OS issue if you go Spotify.

That’s a good idea actually! Have not tried that.

I’ve tried changing so many things in Windows and in the Bios, but I hadn’t tried disabling SMT until today. Usually LatencyMon would report an unacceptable DPC routine execution time within 5-10 minutes (with as many apps/services closed as I could). Post disabling SMT, I’ve been running LatencyMon for almost 3 hours while doing other things on my PC (nothing disabled) and haven’t seen ntoskrnl report higher than 700 microseconds which is WAY better. I don’t really have any use cases for super high thread count, so I’m not at a loss here… but what an odd thing for that to be. I assume there’s some sort of RAM thrashing going on, but… idk.

I’ll keep running it to see what I can get to happen.

For anyone interested, I started googling around for threadripper SMT stutter/latency and stumbled upon this benchmark article: The Windows and Multithreading Problem (A Must Read) - The 64 Core Threadripper 3990X CPU Review: In The Midst Of Chaos, AMD Seeks Opportunity

I’m on Windows 10 Pro and what I’m experiencing falls very much in line with a processor or memory hiccup. I will see disk access latency, random network latency/drops, my audio issues… I’m tempted to try Windows 10 Pro for Workstations, but I really don’t need the extra thread count.

1 Like

Good Luck @TheLegend_TubaGuy

Keep us apprised of your progress. I have a TR 5975wx on the way and plan to use the same Mobo as you. Doubt I’ll have that much RAM but interested to see your solution.

Have not tried Windows for Workstations as I don’t really want to shell out the money, but disabling SMT has completely fixed this issue for me.

3 Likes

Good Stuff. Let me know if you encounter any issues with SMT off. I have a LTS Win10 Enterprise license I was going to use on mine.

WfW might be a long term solution, but I have no direct experience on that version. Just checked and we have a few hundred Win 10 Pro Workstation licenses at work with none in use. Will see if I can talk my boss into lending me a key for “testing” long term. :grin:

I have a similar setup,3975wx 128gb 3200ecc 3080ti a couple of nvme 2tb drives.
I have not had any experience with spotify in particular, but my audio has been rock solid from day one, with the most current bios and very stock bios settings.
I listen to various streams from my local streaming server as well as bandcamp or soundcloud.
Are you using the on board audio, or a USB sound device?

Totally fair and valid question, but note that I ran LatencyMon without anything but my monitors/keyboard/mouse plugged in and still had high DPC latency issues crop up.

Typically I use a GoXLR. But again, even if I unplug this and close the software I’m still seeing high DPC latency problems crop up via LatencyMon. I realize I started this thread citing audio dropouts/glitches, but the reality is I’m experiencing occasional stutter regardless of what I’m doing.

@shadragon
I was too curious and ended up buying 10 pro for workstations, re-enabled SMT and saw high DPC latency via LatencyMon within 10-15 minutes of it running from the kernel.

I’m going to try a fresh windows install with no other peripherals, get it updated, run LatencyMon and see what happens. I’m pulling my hair out trying to track this down.

Update: I just ran LatencyMon for 15 minutes on a fresh windows 10 pro (non-workstation) install (only installed updates/chipset drivers/nvidia driver/intel nic driver) and still get LatencyMon reporting high DPC latency with SMT enabled. I guess my next step is memory testing…? I had a keyboard, mouse, and monitors plugged in. Removed all add-in cards except the 3090ti before going through the install.

I did end up trying to do this but it doesn’t seem like the board will let me do that. The bios flash utility will not accept flashing to the previous version. Not sure if it’s because I updated the “firmware” also or what’s up with that.

Memory ended up all testing okay, zero errors over multiple passes.

I’m again not really sure where to go from here. I might contact ASUS and try to RMA the board.

I have built a significant amount of systems with this board and have not has a single report of any audio stuttering or latency spikes. My customers are using systems for 3D rendering, Hi spec modeling and machine learning amongst other tasks, if someone was going to notice it would have been them. I would suggest RMAing the motherboard as the issues you are experiencing are not normal.

BTW the “firmware” is the IKVM/BMC firmware.

1 Like

As I understand it, pro audio exposes different issues from those compute-type workloads. For a musician, latency is evil.

1 Like

I’m going through advanced replacement with Asus seeing as the only components I haven’t swapped out at this point are CPU, mobo, and RAM… and the RAM is testing fine.

I’ll update when I’ve got that installed.

Yeah I do think the issue I’m running in to most people wouldn’t notice, but the occasional unresponsiveness from clicking something/network drops/nvme drops… Stutter is the best way I can describe it. I’m not sure if most end users would perceive that, but idk.

I would be super curious if anyone with a similar set up has run latencymon. Even just a “no, that doesn’t happen to me” would let me know I’m not expecting something insane :upside_down_face:

2 Likes

Over a month later I finally have some updates. Some good, some bad, some both.

Replaced the motherboard and continued to see issues in latencymon showing up as “interrupt to process latency”. I disabled threading and the latency spikes decreased dramatically, as before, but didn’t go away entirely.

One latency spike showed up as both “interrupt to process latency” and a “highest reported DPC routine execution time” which called out the networking driver. I started googling around this and found the interrupt moderation rate setting for my adapter (built in dual x550). I’m running 10 Gbps and have been basically the entire time I’ve noticed latency so thought that was interesting. I set interrupt moderation rate to Off and all of the checksum offload settings to disabled. I also at one point saw Windows Defender show up as a “Highest reported ISR routine execution time” process so have permanently disabled that while I hunt down what’s going on. I can now run close to 24 hours without latencymon reporting any issues. I think the “Interrupt Moderation Rate” setting is SUPER interesting because the “stutter” I experience feels like a CPU interrupt/lag/something issue that I originally assumed was due to something with PCIe devices. For completeness of this post, I have also disabled the other ethernet controller I’m not using, as well as the on board wifi. I have tested disabling the on board audio and that didn’t make any difference. I have also tested with a different GPU (3080ti) with no change.

But it’s still not perfect. I just stopped a test at close to 17 hours and it showed a spike of almost 7000 microseconds. At the same time I’m running these tests, I have OBS open and streaming a bandwidth test. When I saw latencymon spike OBS also showed that it had dropped connection. When this happens the ethernet controller doesn’t go away, it just loses connection for a moment and then OBS reconnects. I have other software running that crashes when the ethernet controller disappears (found out when changing interrupt moderation rate and checksum offload settings), so I know it’s not doing that.

I’m currently connected through an entirely different switch at 1 Gbps to see if that makes any difference.

As always if anyone has anything else for me to look at, test, try, I’d love to hear to it.

2 Likes

Well at almost 5 hours I ran in to issues similar to what I posted at the end of my last post. I’ve started keeping all these LatencyMon reports to see if I can spot anything between them. This time it was high dpc routine execution time from wdf01000.sys.

LatencyMon Report

I have no idea what to check when the reported dpc routine is the windows kernel. I’ve also tried with half the ram, then the other half, just to check even though it’s all tested fine. No change.