Cyberpunk 2077 low GPU usage and poor performance on PCI Passthrough System

Operating Systems & Open Source Virtualization
1

I’ve been running my unRAID server with a PCI Passthrough gaming VM for about 7 or 8 years now, this particular system for about 6 and had little to no trouble with it.

I recently upgraded my system to a 5900X and a RX 6750 XT (I can post full specs of the server if needed) and decided to try out Cyberpunk 2077. My old system could run the game, but not particularly well, and I got less than stellar framerates. The old system was a Ryzen 1700 and GTX 1080.

Now, when I run the Cyberpunk benchmark, I get right around 20 FPS regardless of settings. Low to Ultra, it all runs the same. But in baremetal windows, I can get 90+ FPS on Ultra settings. What gives? In the VM, I look at task manager and the GPU usage is only like 15-20%, and CPU is only around 40% so its not bottlenecking I don’t think…Something is different between the baremetal system and the VM.

Any ideas where to look? I’ll include the XML for my unRAID gaming VM below to look at. Again, other games all run just great, this is so far the only one I’ve run into that runs bad.

Let me know what else might be useful and I’ll be happy to provide the info!

<?xml version='1.0' encoding='UTF-8'?>
<domain type='kvm' id='5'>
  <name>Windows 10</name>
  <uuid>afb22e68-6160-c234-7de8-a72f6d0501f6</uuid>
  <metadata>
    <vmtemplate xmlns="unraid" name="Windows 10" icon="windows.png" os="windows10"/>
  </metadata>
  <memory unit='KiB'>25165824</memory>
  <currentMemory unit='KiB'>25165824</currentMemory>
  <memoryBacking>
    <nosharepages/>
  </memoryBacking>
  <vcpu placement='static'>12</vcpu>
  <cputune>
    <vcpupin vcpu='0' cpuset='6'/>
    <vcpupin vcpu='1' cpuset='18'/>
    <vcpupin vcpu='2' cpuset='7'/>
    <vcpupin vcpu='3' cpuset='19'/>
    <vcpupin vcpu='4' cpuset='8'/>
    <vcpupin vcpu='5' cpuset='20'/>
    <vcpupin vcpu='6' cpuset='9'/>
    <vcpupin vcpu='7' cpuset='21'/>
    <vcpupin vcpu='8' cpuset='10'/>
    <vcpupin vcpu='9' cpuset='22'/>
    <vcpupin vcpu='10' cpuset='11'/>
    <vcpupin vcpu='11' cpuset='23'/>
  </cputune>
  <resource>
    <partition>/machine</partition>
  </resource>
  <os>
    <type arch='x86_64' machine='pc-i440fx-6.2'>hvm</type>
    <loader readonly='yes' type='pflash'>/usr/share/qemu/ovmf-x64/OVMF_CODE-pure-efi-tpm.fd</loader>
    <nvram>/etc/libvirt/qemu/nvram/afb22e68-6160-c234-7de8-a72f6d0501f6_VARS-pure-efi-tpm.fd</nvram>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv mode='custom'>
      <relaxed state='on'/>
      <vapic state='on'/>
      <spinlocks state='on' retries='8191'/>
      <vendor_id state='on' value='none'/>
    </hyperv>
  </features>
  <cpu mode='host-passthrough' check='none' migratable='on'>
    <topology sockets='1' dies='1' cores='6' threads='2'/>
    <cache mode='passthrough'/>
    <feature policy='require' name='topoext'/>
  </cpu>
  <clock offset='localtime'>
    <timer name='hypervclock' present='yes'/>
    <timer name='hpet' present='no'/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>restart</on_crash>
  <devices>
    <emulator>/usr/local/sbin/qemu</emulator>
    <disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='writeback'/>
      <source dev='/dev/disk/by-id/ata-WD_Blue_SA510_2.5_1000GB_223326801852' index='1'/>
      <backingStore/>
      <target dev='hdc' bus='sata'/>
      <serial>vdisk1</serial>
      <boot order='1'/>
      <alias name='sata0-0-2'/>
      <address type='drive' controller='0' bus='0' target='0' unit='2'/>
    </disk>
    <controller type='virtio-serial' index='0'>
      <alias name='virtio-serial0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x05' function='0x0'/>
    </controller>
    <controller type='sata' index='0'>
      <alias name='sata0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x04' function='0x0'/>
    </controller>
    <controller type='pci' index='0' model='pci-root'>
      <alias name='pci.0'/>
    </controller>
    <controller type='usb' index='0' model='qemu-xhci' ports='15'>
      <alias name='usb'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/>
    </controller>
    <interface type='bridge'>
      <mac address='52:54:00:76:00:5c'/>
      <source bridge='br0.20'/>
      <target dev='vnet4'/>
      <model type='virtio-net'/>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x02' function='0x0'/>
    </interface>
    <serial type='pty'>
      <source path='/dev/pts/4'/>
      <target type='isa-serial' port='0'>
        <model name='isa-serial'/>
      </target>
      <alias name='serial0'/>
    </serial>
    <console type='pty' tty='/dev/pts/4'>
      <source path='/dev/pts/4'/>
      <target type='serial' port='0'/>
      <alias name='serial0'/>
    </console>
    <channel type='unix'>
      <source mode='bind' path='/var/lib/libvirt/qemu/channel/target/domain-5-Windows 10/org.qemu.guest_agent.0'/>
      <target type='virtio' name='org.qemu.guest_agent.0' state='connected'/>
      <alias name='channel0'/>
      <address type='virtio-serial' controller='0' bus='0' port='1'/>
    </channel>
    <input type='tablet' bus='usb'>
      <alias name='input0'/>
      <address type='usb' bus='0' port='4'/>
    </input>
    <input type='mouse' bus='ps2'>
      <alias name='input1'/>
    </input>
    <input type='keyboard' bus='ps2'>
      <alias name='input2'/>
    </input>
    <tpm model='tpm-tis'>
      <backend type='emulator' version='2.0' persistent_state='yes'/>
      <alias name='tpm0'/>
    </tpm>
    <audio id='1' type='none'/>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0c' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev0'/>
      <rom file='/mnt/user/isos/BIOS Dumps/6750XT.rom'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x03' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x0c' slot='0x00' function='0x1'/>
      </source>
      <alias name='hostdev1'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x06' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='pci' managed='yes'>
      <driver name='vfio'/>
      <source>
        <address domain='0x0000' bus='0x07' slot='0x00' function='0x0'/>
      </source>
      <alias name='hostdev2'/>
      <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source startupPolicy='optional'>
        <vendor id='0x046d'/>
        <product id='0xc07c'/>
        <address bus='1' device='2'/>
      </source>
      <alias name='hostdev3'/>
      <address type='usb' bus='0' port='1'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source startupPolicy='optional'>
        <vendor id='0x0c45'/>
        <product id='0x7605'/>
        <address bus='7' device='3'/>
      </source>
      <alias name='hostdev4'/>
      <address type='usb' bus='0' port='2'/>
    </hostdev>
    <hostdev mode='subsystem' type='usb' managed='no'>
      <source startupPolicy='optional' missing='yes'>
        <vendor id='0x047f'/>
        <product id='0xc05e'/>
      </source>
      <alias name='hostdev5'/>
      <address type='usb' bus='0' port='3'/>
    </hostdev>
    <memballoon model='none'/>
  </devices>
  <seclabel type='dynamic' model='dac' relabel='yes'>
    <label>+0:+100</label>
    <imagelabel>+0:+100</imagelabel>
  </seclabel>
</domain>

Thanks in advance!

2

Cyberpunk doesn’t like to apply settings changes until you quit the game and start it back up, so make your changes and restart the game

You likely had ray tracing on, 6750 can do rt as in it has hardware for it, but you really shouldn’t since it doesn’t do it well

If that’s not it, enable rebar?

3

OK this is sort of resolved…of course as usual, I can spend hours upon hours trying everything I can think of with no luck, but as soon as I ask for help, I solve it myself.

I found this thread Low GPU Usage / half Performance of Baremetal! (rtx3090) - pls :( - VM Engine (KVM) - Unraid which suggests adding this line “” to your XML. I did that, and suddenly I’m getting baremetal-like performance again, in the VM.

I don’t know for sure what this line does though. That forum post links to another discussion here on L1T that I’m reading through now KVM parameters to optimize Hyper-V with AMD processors - #3 by vic

Someone there mentioned that that really shouldn’t have made a difference. They mentioned that maybe Hyper-V was enabled in the guest which it shouldn’t have. I’m going to try removing that line and then check to make sure Hyper-V is not enabled on my guest and see if that makes a difference.

If anyone knows of any downsides or caveats to adding this line, please let me know. So far it seems to have made all the difference!

4

I had a similar issue. No matter how many benchmark runs i did, my gpu stuck in a low power mode.
I fixed it by enabling radeon chill in the driver. Set a minimum fps target and a maximum fps target and you should be good.

5

In your setup each virtual device has a different bus/slot, you should use multifunction like explained here

multifunction_pci_devices_to_kvm_guest_virtual_machines

This doesn’t have to be a problem in your case, but e.g with ROCm it can cause problems.
Always adapt the virtual hardware to the physical hardware as closely as possible

6

@Janos thanks for the tip! I’ll be honest, the XML for that machine is about 6 years old. I’ve tweaked it here and there but I know there’s a lot of new advice that’s come out to optimize things since then that I’ve not gotten around to applying. In my research about a different problem, I kept seeing mention of the “multifunction” thing so I’ll have to give that a try.