Windows 10 VM with GPU passthough is epic slow, but Linux VMs with GPU passthrough are just fine

Operating Systems & Open Source Linux
,
1

I’m not sure what’s going on, or how to find the necessary logs to track this down, but for some reason, the Windows VM is slowing down like crazy, but the Linux VMs (Ubuntu Unity, KDE Neon, Arch, etc) are perfectly fine.

Host:

Operating System: Kubuntu 22.10
KDE Plasma Version: 5.27.3
KDE Frameworks Version: 5.104.0
Qt Version: 5.15.6
Kernel Version: 6.2.8-1-liquorix-amd64 (64-bit)
Graphics Platform: X11
Processors: 24 × AMD Ryzen 9 5900X 12-Core Processor
Memory: 62.7 GiB of RAM
Graphics Processor: AMD Radeon RX 6900 XT
Manufacturer: Gigabyte Technology Co., Ltd.
Product Name: X570S AORUS MASTER
System Version: -CF

The guest GPU is an RTX 3060 Ti.

Windows 10 config:

<domain type="kvm">
  <name>Windows</name>
  <uuid>6440ba51-6199-451e-b6ed-bd5b4e6dc039</uuid>
  <metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">25165824</memory>
  <currentMemory unit="KiB">25165824</currentMemory>
  <vcpu placement="static">12</vcpu>
  <os>
    <type arch="x86_64" machine="pc-q35-6.0">hvm</type>
    <loader readonly="yes" type="pflash">/usr/share/OVMF/OVMF_CODE_4M.ms.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/Windows_VARS.fd</nvram>
    <bootmenu enable="no"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode="custom">
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
      <vendor_id state="on" value="AuthenticAMD"/>
    </hyperv>
    <vmport state="off"/>
  </features>
  <cpu mode="host-passthrough" check="none" migratable="on">
    <topology sockets="1" dies="1" cores="6" threads="2"/>
    <feature policy="disable" name="hypervisor"/>
  </cpu>
  <clock offset="localtime">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2" discard="unmap"/>
      <source file="/home/DATA/870E4TB/yamiyuki/VMs/Windows.qcow2"/>
      <target dev="sda" bus="sata"/>
      <boot order="1"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <disk type="file" device="cdrom">
      <driver name="qemu" type="raw"/>
      <source file="/home/DATA/SGBC2T/yamiyuki/Downloads/ISOs/virtio-win-0.1.229.iso"/>
      <target dev="sdb" bus="sata"/>
      <readonly/>
      <address type="drive" controller="0" bus="0" target="0" unit="1"/>
    </disk>
    <disk type="block" device="disk">
      <driver name="qemu" type="raw" cache="none" io="native" discard="unmap"/>
      <source dev="/dev/disk/by-label/WinVM_DATA"/>
      <target dev="sdg" bus="sata"/>
      <address type="drive" controller="1" bus="0" target="0" unit="0"/>
    </disk>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/home/yamiyuki/VMs/VM Data/WinGames.qcow2"/>
      <target dev="sdh" bus="sata"/>
      <address type="drive" controller="1" bus="0" target="0" unit="1"/>
    </disk>
    <controller type="sata" index="0">
      <address type="pci" domain="0x0000" bus="0x00" slot="0x1f" function="0x2"/>
    </controller>
    <controller type="sata" index="1">
      <address type="pci" domain="0x0000" bus="0x07" slot="0x01" function="0x0"/>
    </controller>
    <controller type="pci" index="0" model="pcie-root"/>
    <controller type="pci" index="1" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="1" port="0x10"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="2" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="2" port="0x11"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x1"/>
    </controller>
    <controller type="pci" index="3" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="3" port="0x12"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x2"/>
    </controller>
    <controller type="pci" index="4" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="4" port="0x13"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x3"/>
    </controller>
    <controller type="pci" index="5" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="5" port="0x14"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x4"/>
    </controller>
    <controller type="pci" index="6" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="6" port="0x15"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x5"/>
    </controller>
    <controller type="pci" index="7" model="pcie-to-pci-bridge">
      <model name="pcie-pci-bridge"/>
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0"/>
    </controller>
    <controller type="pci" index="8" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="8" port="0x16"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x02" function="0x6"/>
    </controller>
    <controller type="pci" index="9" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="9" port="0x8"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0" multifunction="on"/>
    </controller>
    <controller type="pci" index="10" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="10" port="0x9"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x1"/>
    </controller>
    <controller type="pci" index="11" model="pcie-root-port">
      <model name="pcie-root-port"/>
      <target chassis="11" port="0xa"/>
      <address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x2"/>
    </controller>
    <controller type="virtio-serial" index="0">
      <address type="pci" domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
    </controller>
    <controller type="usb" index="0" model="qemu-xhci" ports="15">
      <address type="pci" domain="0x0000" bus="0x02" slot="0x00" function="0x0"/>
    </controller>
    <interface type="network">
      <mac address="52:54:00:3f:41:42"/>
      <source network="default"/>
      <model type="e1000e"/>
      <address type="pci" domain="0x0000" bus="0x09" slot="0x00" function="0x0"/>
    </interface>
    <serial type="pty">
      <target type="isa-serial" port="0">
        <model name="isa-serial"/>
      </target>
    </serial>
    <console type="pty">
      <target type="serial" port="0"/>
    </console>
    <channel type="spicevmc">
      <target type="virtio" name="com.redhat.spice.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="2"/>
    </channel>
    <channel type="spiceport">
      <source channel="org.spice-space.webdav.0"/>
      <target type="virtio" name="org.spice-space.webdav.0"/>
      <address type="virtio-serial" controller="0" bus="0" port="1"/>
    </channel>
    <input type="mouse" bus="ps2"/>
    <input type="keyboard" bus="virtio">
      <address type="pci" domain="0x0000" bus="0x08" slot="0x00" function="0x0"/>
    </input>
    <input type="mouse" bus="usb">
      <address type="usb" bus="0" port="1"/>
    </input>
    <input type="keyboard" bus="ps2"/>
    <tpm model="tpm-tis">
      <backend type="emulator" version="2.0"/>
    </tpm>
    <graphics type="spice">
      <listen type="none"/>
      <image compression="off"/>
    </graphics>
    <audio id="1" type="spice"/>
    <video>
      <model type="none"/>
    </video>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x03" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x06" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x0e" slot="0x00" function="0x0"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x04" slot="0x00" function="0x0"/>
    </hostdev>
    <hostdev mode="subsystem" type="pci" managed="yes">
      <source>
        <address domain="0x0000" bus="0x0e" slot="0x00" function="0x1"/>
      </source>
      <address type="pci" domain="0x0000" bus="0x05" slot="0x00" function="0x0"/>
    </hostdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="2"/>
    </redirdev>
    <redirdev bus="usb" type="spicevmc">
      <address type="usb" bus="0" port="3"/>
    </redirdev>
    <memballoon model="none"/>
    <shmem name="looking-glass">
      <model type="ivshmem-plain"/>
      <size unit="M">32</size>
      <address type="pci" domain="0x0000" bus="0x07" slot="0x02" function="0x0"/>
    </shmem>
  </devices>
</domain>

I saw this one line in the journal, but I haven’t seen it repeated:

Mar 23 23:00:25 kernel: kvm: SMP vm created on host with unstable TSC; guest TSC will not be reliable
2

Only 2 things i really notice.

  1. Your VM HD and NIC should be on VIRTIO. this has a pretty good performance uplift on windows VMs.

  2. That error you mention reminds me of a old kernel bug. Try this and see what the output is:

# grep "time" /var/log/dmesg
3

out of interest does it still happen with a mainline kernel?

all of the cooked gaming/etc uber kernels have been trash in my opinion

1 Like
4 
$ sudo grep "time" /var/log/dmesg
[    0.357513] kernel: Calibrating delay loop (skipped), value calculated using timer frequency.. 7400.21 BogoMIPS (lpj=3700107)
[    0.608316] kernel: RAPL PMU: API unit is 2^-32 Joules, 1 fixed counters, 163840 ms ovfl timer
[    0.608728] kernel: workingset: timestamp_bits=40 max_order=24 bucket_order=0
[    0.616811] kernel: Estimated ratio of average max frequency by base frequency (times 1024): 1197
[    0.631621] kernel: nvme nvme0: Shutdown timeout set to 10 seconds
[    2.058481] kernel: amdgpu 0000:0d:00.0: amdgpu: PSP runtime database doesn't exist
[    2.058483] kernel: amdgpu 0000:0d:00.0: amdgpu: PSP runtime database doesn't exist
[    5.064152] kernel: amdgpu 0000:0d:00.0: amdgpu: Using BACO for runtime pm
[    5.957249] systemd-journald[1335]: Received client request to flush runtime journal.
[    6.254806] kernel: Bluetooth: hci0: Firmware timestamp 2022.28 buildtype 1 build 47489

I’ll give it a spin with the 5.19 default Ubuntu kernel.

I haven’t booted my VM in about a month (or whenever season 3 of Halo Infinite came out), so I don’t know when it started. I have been using Liquorix for about a while now, though.

5

Well, I tried with default Ubuntu kernel, and it works fine. Weird…

1 Like
6

yep lol

i mever use peoples cooked kernels, they are always dog shit

7

it is sad that you feel this way however, there can be different results depending on what hardware you use and which distro you try it on.

6.2.8-zen1-1-zen on garuda dragonized(arch-based)

8

the evidence is just too strong (see above) as to why they are to be avoided

9

comparing ubuntu to arch(or any other distro) is fuzzy at best. unless they use the same exact binaries there will always be differences.