VFIO/QEMU/libvirt - Arch host, Windows 10 guest, audio crackles

My setup (details below) passes a GPU and NIC straight through to QEMU as well as passing the keyboard and mouse via evdev. I have a speaker amp plugged into my monitor, so am technically using the monitor’s DAC.

Once I fire up the VM, I just switch inputs on my monitor from DisplayPort (via the RX 580 for Linux) to HDMI (via the GTX 1080 Ti for Windows) and the guest has full control.

Everything works surprisingly well, to be honest. I’m really pleased with it. The only issue is that, seemingly at random, I’ll develop the classic issue of the audio crackles. The whole system remains otherwise responsive, but in order to get the audio to behave again, I have to reboot the VM. This happens whether I stay in the VM or switch back to the host for a while, or any combination of movements.

So far, I’ve noticed absolutely nothing in qemu/win10.log that isn’t starting and stopping the VM. I also enabled debug logging for libvirtd, but didn’t see anything overt in there immediately after the audio became crackly.

I will happily post any further details you may require and very much look forward to any feedback!

Thanks in advance, everyone.

Hardware

  • Threadripper 2950X
  • 32GB RAM
  • ASRock X399 Taichi (P3.50, Channel memory interleaving)
  • Sapphire Nitro+ RX 580
  • EVGA GTX 1080 Ti SC2 Black Edition

Host Software

  • 5.3.1-arch1-1-ARCH
  • libvirt 5.6.0-1
  • ovmf 1:r26214.20d2e5a125-1
  • qemu 4.1.0-1

Guest Software

  • Windows 10 1903 (Performance power profile)
  • Nvidia 436.3

Guest XML
(I tried posting this, but couldn’t get the parser to leave the XML alone - any help on having it ignore open/close angle braces?)

GPU lspci

$ sudo lspci -kvv -s 08:00.0
08:00.0 VGA compatible controller: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: eVga.com. Corp. GP102 [GeForce GTX 1080 Ti]
	Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- 
	Capabilities: [420 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
		AERCap:	First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
			MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
		HeaderLog: 00000000 00000000 00000000 00000000
	Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
	Capabilities: [900 v1] Secondary PCI Express <?>
	Kernel driver in use: vfio-pci
	Kernel modules: nouveau

HDMI Audio lspci

$ sudo lspci -kvv -s 08:00.1
08:00.1 Audio device: NVIDIA Corporation GP102 HDMI Audio Controller (rev a1)
	Subsystem: eVga.com. Corp. GP102 HDMI Audio Controller
	Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-
	Interrupt: pin B routed to IRQ 137
	NUMA node: 0
	Region 0: Memory at d9080000 (32-bit, non-prefetchable) [size=16K]
	Capabilities: [60] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
		Status: D3 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [68] MSI: Enable- Count=1/1 Maskable- 64bit+
		Address: 0000000000000000  Data: 0000
	Capabilities: [78] Express (v2) Endpoint, MSI 00
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0.000W
		DevCtl:	CorrErr- NonFatalErr- FatalErr- UnsupReq-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <1us, L1 <4us
			ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM Disabled; RCB 64 bytes Disabled- CommClk+
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-
		LnkSta:	Speed 8GT/s (ok), Width x16 (ok)
			TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range AB, TimeoutDis+, LTR+, OBFF Via message
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-, LTR-, OBFF Disabled
			 AtomicOpsCtl: ReqEn-
		LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete-, EqualizationPhase1-
			 EqualizationPhase2-, EqualizationPhase3-, LinkEqualizationRequest-
	Capabilities: [100 v2] Advanced Error Reporting
		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UEMsk:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
		UESvrt:	DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
		CEMsk:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
		AERCap:	First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
			MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
		HeaderLog: 00000000 00000000 00000000 00000000
	Kernel driver in use: vfio-pci
	Kernel modules: snd_hda_intel

Are you using the HDMI or DisplayPort on the 1080 Ti for audio? If so, you can look into enabling Message Signaled Interrupts. Fixed the problem for me, though it should be noted that I was using an ESXi VM and a 1050 Ti.

1 Like

If you are using looking glass and the audio is cracking with the AC97 driver, that happens

Take a look at scream

In my vfio setup to reduce audio latency I passthrough the motherboard main sound device to the Windows vm & have a cheap pcie sound card I use for Linux (the CMI8738/CMI8768 cards on Amazon that do not work on Windows 10 properly work fine in Linux).

Setting iothreadpin &emulatorpin to different cores helped a little too.

The seamless vm audio setup here is a nice hardware solution.