Return to Level1Techs.com

High DPC latency + Audio Crackle

Hello I am new to VFIO but not new to Linux. First I will post Specs then Configs/Logs

Things I’ve tried:
Pinned CPUs cores 2-7
Used isolcpus
Recompiled Manjaro Kernel with VFIO patches
Installed VirtIO drivers for all devices and Device Manager
Used MSI v2 tool to enable MSI for GPU and GPU Audio

OS:Manjaro
Kernel: Personally Compiled 5.6.11-1 with VFIO Patches.

Specs:
Gigabyte Z390 AORUS Motherboard
i7-9700KF 5.0GHz @1.35V (NOT multithreaded)
EVGA 2080 Super XC HYBRID (Passed to VM)
GT 210 ( Arch Linux Host)
EVGA 650W Gold+ PSU

VM XML:

<domain type="kvm">
  <name>win10</name>
  <uuid>26314a1c-8d6c-4639-b296-2dde5ef4377b</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">12386304</memory>
  <currentMemory unit="KiB">12386304</currentMemory>
  <memoryBacking>
    <hugepages/>
  </memoryBacking>
  <vcpu placement="static" current="6">36</vcpu>
  <iothreads>1</iothreads>
  <cputune>
    <vcpupin vcpu="0" cpuset="1"/>
    <vcpupin vcpu="1" cpuset="5"/>
    <vcpupin vcpu="2" cpuset="2"/>
    <vcpupin vcpu="3" cpuset="6"/>
    <vcpupin vcpu="4" cpuset="3"/>
    <vcpupin vcpu="5" cpuset="7"/>
    <emulatorpin cpuset="0,4"/>
  </cputune>
  <os>
    <type arch="x86_64" machine="pc-q35-4.2">hvm</type>
    <loader readonly="yes" type="pflash">/usr/share/edk2-ovmf/x64/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10_VARS.fd</nvram>
    <bootmenu enable="yes"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vendor_id state="on" value="anything"/>
    </hyperv>
    <kvm>
      <hidden state="on"/>
    </kvm>
    <vmport state="off"/>
    <ioapic driver="kvm"/>
  </features>
  <cpu mode="host-passthrough" check="none">
    <topology sockets="1" cores="6" threads="6"/>
    <feature policy="require" name="invtsc"/>
    <feature policy="require" name="topoext"/>
  </cpu>
  <clock offset="utc">
    <timer name="tsc" present="yes" mode="native"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/home/ech0/Downloads/Microsoft_Windows_10_Enterprise_Version_1903_X64-CYGiSO/cyg-w1903x64en.iso"/>
      <target dev="sda" bus="sata"/>
      <readonly/>
      <boot order="1"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/home/ech0/Downloads/virtio-win-0.1.171.iso"/>
      <target dev="sdb" bus="sata"/>
      <readonly/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/var/lib/libvirt/images/win10.qcow2"/>
      <target dev="vda" bus="virtio"/>
      <boot order="2"/>
      <address type="pci" domain="0x0000" bus="0x0b" slot="0x00" function="0x0"/>
    </disk>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="7" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0x17"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x7"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x18"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="10" model="pcie-to-pci-bridge">
      <model name="pcie-pci-bridge"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="11" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="11" port="0x19"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x03" function="0x1"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <controller type="scsi" index="0" model="virtio-scsi">
      <address type="pci" domain="0x0000" bus="0x07" slot="0x00" function="0x0"/>
    </controller>
    <controller type="scsi" index="1" model="virtio-scsi">
      <address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
    </controller>
    <interface type="network">
      <mac address="52:54:00:bd:69:df"/>
      <source network="br1"/>
      <model type="virtio"/>
      <link state="up"/>
      <address type="pci" domain="0x0000" bus="0x09" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <input type="tablet" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="ps2"/>
    <graphics type="spice" autoport="yes">
      <listen type="address"/>
      <image compression="off"/>
      <gl enable="no"/>
    </graphics>
    <video>
      <model type="qxl" ram="65536" vram="65536" vgamem="16384" heads="1" primary="yes"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
    </video>
    <hostdev mode="subsystem" type="usb" managed="yes">
      <source>
        <vendor id="0x046d"/>
        <product id="0xc539"/>
      </source>
      <address type="usb" bus="0" port="2"/>
    </hostdev>
    <hostdev mode="subsystem" type="usb" managed="yes">
      <source>
        <vendor id="0x0b05"/>
        <product id="0x184d"/>
      </source>
      <address type="usb" bus="0" port="3"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x01" slot="0x00" function="0x1"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="usb" managed="yes">
      <source>
        <vendor id="0x1852"/>
        <product id="0x7022"/>
      </source>
      <address type="usb" bus="0" port="4"/>
    </hostdev>
    <memballoon model="virtio">
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </memballoon>
  </devices>
</domain>

Grub Boot Line
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt isolcpus=2-7 nohz_full=2-7 rcu_nocbs=2-7 transparent_hugepage=never rd.driver.pre=vfio-pci vfio-pci.ids=10de:1e81,10de:10f8 pcie_acs_override=downstream,mul>

Passed Through Devices:
2080 Super
Nvidia HD Audio Controller (GPU Audio)
Logitech Wireless Pro
YROCOM C&C Co., LTD DigiHug USB Audio (Fii) Olypus 2 AMP + HD6xx headphones)

I’ve tried Plugging the USB DAC for my headset into the dedicated USB-DAC port on my Mobo as well as a standard USB-3.0 port

Ok still doing some testing but I think passing the onboard XHCI 3.1 USB Controller to the VM fixed audio issues. Not ideal but I guess it’s that or getting a separate soundcard to take up my last PCI Slot.

Or I could set this up Scream Virtual Audio Card/Server as a last resort but I’d rather not

Still would like some advice on improving the High DPC Latency tho!

If you experience high dpc latency issues on windows,
then it could be a driver related issue.
You could use a tool like latency mon, that might give you some clue’s,
where to start looking.

I assume you run the latest bios already?

I actually had already used Latencymon as that is how I fouund out about this issues.

I think it hinted at the nvidia driver or the Nvidia Audio driver (even tho I am using a USB3 AMP+ Headset for audio)

I am not on the latest BIOS actually. I didn’t bother updating to the latest as I thought the only reason to update was to fix IOMMMU groupings but I read that IOMMU grouping was screwed on this Board anyways so I didn’t bother.

After I passed my XHCI 3.1 controller to the VM I played COD MW Moshpit online and the SIngleplayer Compaign and has zero issues. Just annoyed with the values in LatancyMod

Still really high DPC Latency. I have modified my grubline a bit
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt isolcpus=2-7 nohz_full=2-7 rcu_nocbs=2-7 kvm.ignore_msrs=1 transparent_hugepage=never rd.driver.pre=vfio-pci vfio-pci.ids=10de:1e81,10de:10f8 pcie_acs_override=downstream,multifunction apparmor=1 security=apparmor udev.log_priority=3>"

No audio issues since passing through USB Controller

Can someone check my XML to make sure I am passing my Cores correctly

My Numa layout
https://justpaste.it/46ssq

EDIT 1: OK I played around with the CPU part of the XML and I am getting higher R20 CPU Scores now. Got 3700 for MultiCore test for my [email protected]

That seems to be about what i should get. Yet my FPS in games seems to be down aboout 25% or so.

For Example in BFV at Ultra Settings (No RayTracing): 80-110FPS

<vcpu placement="static" current="8">16</vcpu>
  <iothreads>1</iothreads>
  <cputune>
    <vcpupin vcpu="0" cpuset="2"/>
    <vcpupin vcpu="1" cpuset="3"/>
    <vcpupin vcpu="2" cpuset="4"/>
    <vcpupin vcpu="3" cpuset="5"/>
    <vcpupin vcpu="4" cpuset="6"/>
    <vcpupin vcpu="5" cpuset="7"/>
    <emulatorpin cpuset="0-1"/>
  </cputune>