Return to Level1Techs.com

VFIO tuning on Threadripper 3960x

Hello all,

I’m looking for advice on tuning and performance enhancing for VFIO (2 Machines, 1 CPU) sort. I have a working setup, with Fedora 34, following the guides here on level1. https://forum.level1techs.com/t/fedora-33-ultimiate-vfio-guide-for-2020-2021-wip/163814/4 and I’, seeing rather intense stuttering and almost unplayable gaming play at times, not all the time, just some of the time. Below is a list of the hardware/software and general layout.

Hardware:Entire Build listed: heller64bit - Saved Part Lists - PCPartPicker

Host Hardware:
-Samsung 970 Evo 1TB (Fedora 34 Linux)
-GTX 1080 Turbo is the Host (Linux) GPU.
-Gigabyte TRX40 Aorus Master EATX MOBO
-G.Skill 4x32GB DDR4-3600 CL18 RAM

Windows Guest Passtrhough:
-32GB Allocated to the VM.
-Onboard Intel GiB NIC
-XFX 6800XT
-PCIe USB card (for native USB plug-n-play)
-Sabrent Rocket 4 Plus (Native install of windows)

Software:-Fedora 34 Linux-VFIO (Virtual Function I/O, Native built-in)
-KVM (Kernel Virtual Machine, built-in to the Linux Kernel)
-QEMU (Open-source Hypervisor, CLI)-Libvirt (Open Source API to manage hypervisors, like QEMU, VMWare, etc…)
-Virt-Manager (GUI Application to manage all the above software pieces.)

Below is the XML, I’ve not set CPU pinning as it doesn’t seam to help.

<metadata>
    <libosinfo:libosinfo xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
      <libosinfo:os id="http://microsoft.com/win/10"/>
    </libosinfo:libosinfo>
  </metadata>
  <memory unit="KiB">33554432</memory>
  <currentMemory unit="KiB">33554432</currentMemory>
  <vcpu placement="static">12</vcpu>
  <os>
    <type arch="x86_64" machine="pc-q35-5.2">hvm</type>
    <loader readonly="yes" type="pflash">/usr/share/edk2/ovmf/OVMF_CODE.fd</loader>
    <nvram>/var/lib/libvirt/qemu/nvram/win10-2_VARS.fd</nvram>
    <boot dev="hd"/>
    <bootmenu enable="yes"/>
  </os>
  <features>
    <acpi/>
    <apic/>
    <hyperv>
      <relaxed state="on"/>
      <vapic state="on"/>
      <spinlocks state="on" retries="8191"/>
    </hyperv>
    <vmport state="off"/>
  </features>
  <cpu mode="host-model" check="partial">
    <topology sockets="1" dies="1" cores="6" threads="2"/>
  </cpu>
  <clock offset="localtime">
    <timer name="rtc" tickpolicy="catchup"/>
    <timer name="pit" tickpolicy="delay"/>
    <timer name="hpet" present="no"/>
    <timer name="hypervclock" present="yes"/>
  </clock>
  <on_poweroff>destroy</on_poweroff>
  <on_reboot>restart</on_reboot>
  <on_crash>destroy</on_crash>
  <pm>
    <suspend-to-mem enabled="no"/>
    <suspend-to-disk enabled="no"/>
  </pm>
  <devices>
    <emulator>/usr/bin/qemu-system-x86_64</emulator>
    <disk type="file" device="disk">
      <driver name="qemu" type="qcow2"/>
      <source file="/var/lib/libvirt/images/win10-2.qcow2"/>
      <target dev="sda" bus="sata"/>
      <address type="drive" controller="0" bus="0" target="0" unit="0"/>
    </disk>
    <disk type="block" device="disk">
      <driver name="qemu" type="raw" cache="writeback"/>
      <source dev="/dev/sdb"/>
      <target dev="hda" bus="sata"/>
      <address type="drive" controller="0" bus="0" target="0" unit="4"/>
    </disk>

try to passthrough a real nic…the default nat interface e1000 is buggy af…at least for me

1 Like

I’ve always used the VirtIO nic with decent success. As an added bonus it means when installing Windows in the VM it’s not connected to the internet, so you can actually create your local account easily and start disabling all the telemetry / un-wanted services prior to enabling the nic when you install the VirtIO driver after the installation.

1 Like

you on fedora 34? guess not

Nope I’m using Arch, shouldn’t really make any difference though.

So I should clarify that I am passing the onboard Intel NIC to the VM as the board has 1 x Aquantia 5GbE LAN chip (5 Gbit/2.5 Gbit/1 Gbit/100 Mbit) (5GLAN) and 1 x Intel® GbE LAN chip (10/100/1000 Mbit) (LAN).

I wonder if it has something to do with CPU scheduling?

for me i was the e1000 issue was fixed with 1. Fedora 34
2. Fedora Updates x86_64
3. kernel-5.14.10-200.fc34.x86_64.rpm

kernel-5.14.10-200.fc34.x86_64.rpm

  • e100e: Fix packet loss issues seen on Intel network controller (Mark Pearson)

check what kernel you are on. Maybe even change to passthrough to the Aquantia instead of the Intel…

Any particular reason / use-case that you have for passing in the Nic that the VirtIO doesn’t cover? Just curious, since it doesn’t really make any sense to me.

because it may depend on the driver used by fedora…also did you update your kernel?