KVM parameters to optimize Hyper-V with AMD processors

Operating Systems & Open Source Virtualization
, ,
1

I was playing PUBG under a W10 Virtual Machine, until recently, where I am getting kicked for using VM. I contacted the support and they have no idea… After a bit of googling, I found some XMLs around which users are able to play.

The only setting that allows me to play actually, is this

<feature policy="disable" name="hypervisor"/>

and then install Hyper-V under Windows. That alone, makes PUBG run. However, when I do that, performance in PUBG is worse, BF2042 also, and other games are unplayable. I ran out of curiosity Superposition 1080 Xtreme and my score dropped from ~11K to 1800 (!!!).

So, I resumed googling for parameters and ended up in a YouTuber that is able to play even more games, which normally do not run under VMs, like Rainbow 6.

I tried to copy all his parameters to my VM, which runs, but soon after I get to desktop, it resets my system, not just W10, but even host system (Manjaro Linux) like I press reset button.

Does any of you can explain, or give me a wiki link, to understand what these settings are and what affects performance with Hyper-V ?

Here is the XML I ended up, but fails to run: <domain type="kvm"> <name>win10</name> <uuid>9cc0be25...</uuid> <metada - Pastebin.com

inxi -F

System:
  Host: wizzy-am5-manjaro Kernel: 6.1.31-2-MANJARO arch: x86_64 bits: 64
    Desktop: KDE Plasma v: 5.27.5 Distro: Manjaro Linux
Machine:
  Type: Desktop System: Gigabyte product: X670E AORUS MASTER v: -CF
    serial: <superuser required>
  Mobo: Gigabyte model: X670E AORUS MASTER v: x.x
    serial: <superuser required> UEFI: American Megatrends LLC. v: F10d
    date: 05/05/2023
CPU:
  Info: 16-core model: AMD Ryzen 9 7950X3D bits: 64 type: MT MCP cache:
    L2: 16 MiB
  Speed (MHz): avg: 3238 min/max: 3000/5759 cores: 1: 3000 2: 2885 3: 3000
    4: 4200 5: 3000 6: 3000 7: 3000 8: 3000 9: 3000 10: 4446 11: 3000 12: 3000
    13: 3000 14: 3000 15: 4200 16: 3000 17: 2883 18: 2885 19: 3000 20: 3354
    21: 2885 22: 4200 23: 3000 24: 2882 25: 3598 26: 4200 27: 3000 28: 2759
    29: 3000 30: 2880 31: 3159 32: 4200
Graphics:
  Device-1: AMD Navi 21 [Radeon RX 6800/6800 XT / 6900 XT] driver: vfio-pci
    v: N/A
  Device-2: AMD Raphael driver: amdgpu v: kernel
  Device-3: Logitech HD Pro Webcam C920 driver: snd-usb-audio,uvcvideo
    type: USB
  Display: x11 server: X.Org v: 21.1.8 with: Xwayland v: 23.1.1 driver: X:
    loaded: amdgpu unloaded: modesetting,radeon dri: radeonsi gpu: amdgpu
    resolution: 1: 1200x1920~60Hz 2: 2560x1440~60Hz
  API: OpenGL v: 4.6 Mesa 23.0.4 renderer: AMD Radeon Graphics (gfx1036
    LLVM 15.0.7 DRM 3.49 6.1.31-2-MANJARO)
Audio:
  Device-1: AMD Navi 21/23 HDMI/DP Audio driver: vfio-pci
  Device-2: AMD Rembrandt Radeon High Definition Audio driver: snd_hda_intel
  Device-3: AMD Family 17h/19h HD Audio driver: snd_hda_intel
  Device-4: Creative Sound Blaster X5
    driver: cdc_acm,hid-generic,snd-usb-audio,usbhid type: USB
  Device-5: Logitech HD Pro Webcam C920 driver: snd-usb-audio,uvcvideo
    type: USB
  API: ALSA v: k6.1.31-2-MANJARO status: kernel-api
  Server-1: PulseAudio v: 16.1 status: active
Network:
  Device-1: Intel Ethernet I225-V driver: igc
  IF: enp15s0 state: up speed: 1000 Mbps duplex: full mac: 74:56:3c:4b:74:7e
  Device-2: Intel Wi-Fi 6 AX210/AX211/AX411 160MHz driver: iwlwifi
  IF: wlp16s0 state: down mac: 06:23:6d:0d:b4:f7
Bluetooth:
  Device-1: Intel AX210 Bluetooth driver: btusb type: USB
  Report: rfkill ID: hci0 rfk-id: 0 state: down bt-service: enabled,running
    rfk-block: hardware: no software: yes address: see --recommends
Drives:
  Local Storage: total: 5.52 TiB used: 1.12 TiB (20.3%)
  ID-1: /dev/nvme0n1 vendor: Kingston model: SKC3000D2048G size: 1.86 TiB
  ID-2: /dev/nvme1n1 vendor: Seagate model: XPG GAMMIX S50 Lite
    size: 953.87 GiB
  ID-3: /dev/nvme2n1 vendor: Samsung model: SSD 970 EVO 500GB
    size: 465.76 GiB
  ID-4: /dev/sda vendor: Crucial model: CT2000BX500SSD1 size: 1.82 TiB
  ID-5: /dev/sdb vendor: Samsung model: SSD 850 EVO 500GB size: 465.76 GiB
Partition:
  ID-1: / size: 448.43 GiB used: 34.59 GiB (7.7%) fs: ext4 dev: /dev/nvme2n1p2
  ID-2: /boot/efi size: 299.4 MiB used: 312 KiB (0.1%) fs: vfat
    dev: /dev/nvme2n1p1
Swap:
  ID-1: swap-1 type: partition size: 8.8 GiB used: 25.5 MiB (0.3%)
    dev: /dev/nvme2n1p3
Sensors:
  System Temperatures: cpu: 41.9 C mobo: N/A gpu: amdgpu temp: 34.0 C
  Fan Speeds (RPM): N/A
Info:
  Processes: 553 Uptime: 4h 37m Memory: available: 61.95 GiB
  used: 41.16 GiB (66.4%) Shell: Zsh inxi: 3.3.27

Kernel parameters:

GRUB_CMDLINE_LINUX_DEFAULT="resume=UUID=eb319a47-23e2-4b2b-ad27-4924407771e0 udev.log_priority=3 amd_iommu=force_enable iommu=pt amdgpu.sg_display=0 vfio-pci.ids=1002:73bf,1002:ab28,10ec:8161 hugepages=16384 systemd.unified_cgroup_hierarchy=1 kvm.ignore_msrs=1 video=efifb:off,vesafb:off pcie_acs_override=downstream,multifunction vfio_iommu_type1.allow_unsafe_interrupts=1
2

Adding some more info…

I was making changes bit-by-bit to find out which parameter causes the reset and I found it:

<feature policy="require" name="x2apic"/>

So, after removing this line, no more resets, and I could play and run benchmarks.

However, performance was still very bad. Again, I was changing one by one the settings, until I am happy with it.

When I changed this:

<feature policy="required" name="svm"/>

to this

<feature policy="disable" name="svm"/>

My performance was back!!!

I was so happy, until I crash landed again… Disabling SVM, brings back performance, but I am getting kicked from certain games, like without Hyper-V enabled…

What am I missing here…?

1 Like
3

This ‘hypervisor’ flag caught my attention. For the past few years, I’ve always had this flag turned on for all VMs. I just realized disabling it for Windows 11 guest improves all-core performance by 1.5-2%. The effect of this flag is consistently reproducible for me. Puzzling me but interesting re-discovery.

The effect of this flag should be minimal if number of virtual cores is fewer than 32. If you find it helpful, it helps then. I always find it tedious to deal these flags with minimal documentation.

I’m quite sure this flag is irrelevant to the performance of the immediate guest. One possible way it might have helped you because you accidentally enabled Hyper-V in your Windows installation, which you shouldn’t have enabled.

4

I installed on purpose Hyper-V in Windows, since this allows me to play certain games, like PUBG, that otherwise block me because their anti-cheat thinks VM is cheating.

The problem is, that with hypervison disabled and Hyper-V enabled, I can play certain games, but the performance is bad.

With hypervisor enabled and without Hyper-V, my performance is 95% of bare-metal Windows, but certain games kick me.

And also, I discovered yesterday, with hypervisor and svm disabled, even though I have Hyper-V installed, performance is great, but I am still getting kicked.

5

I’m not surprised because you’re running nested VMs.

95% is about right. I forgot what percentages when I did the check a long while ago.

The dangling problem that irritated me ever since was that running Cinebench in macOS VM is a few percentage points higher than the same in Windows VM.

Up until yesterday, I didn’t realise the issue is the ‘hypervisor’ flag. Now both my Win and MacOS VMs are on par in terms of Cinebench performance.

I’m quite happy with this consistency. One less irritation bothering me now. lol

It’s unchartered water as I’m not into games. Perhaps run your games in bare-metal instead. It’s also 5% faster, which sounds like the worthiness of a GPU upgrade.

6

Ain’t gonna happen. I prefer to stop playing certain games, than go back to Windows, or even dual-boot.
I am just trying to find a solution, as there are people who run with Hyper-V enabled and they don’t have performance impact. Even RedHat in their Wiki say “almost no performance impact” for nested VM, but I am not there yet.