Hi Everyone.
I have a win10 guest VM on proxmox that I’ve GPU passthroughed to, but the frametimes are rough and the framerates are a bit low. Still above 60. I have followed this guide to get to where I am now: PCI Passthrough - Proxmox VE
I am looking for input from my esteemed vfio peers as to what to do next.
Frametime example and problem explanation
In games I have consistently “decent” framerates, but every say, 30 seconds to a minute or so I will have a rather large frametime jump where it’s quite noticable that it’s “stuttered” and the frametimes are in the 16-20ms range for some games, where as its normally a bit lower. It’s not very smooth either as you can see.
I haven’t yet got to benchmarking the framerate to see if I am getting the same, but I feel that the RX 6900 XT is producing lower framerates then baremetal Win11 as well (my previous setup)
Really I am just concerend about smoothing out this minor stuttering. I can accept “only” 170 frames instead of say 200.
Host Specs, GRUB Contents & Proxmox Version
Grub contents: GRUB_CMDLINE_LINUX_DEFAULT=“quiet intel_iommu=on iommu=pt”
^Just noticed this is INTEL IOMMU, switching it to AMD after posting this… probably wont help…
CPU: Ryzen 3900x
RAM: 64GB 3200mhz Corsair 15-15-15-36 (2x32GB Sticks)
MB: TUF GAMING B550M-PLUS
GPU: RX 6900XT (Navi 10, no reset bug)
GPU: Intel A380 (Unused and currently not showing up in lspci, to investigate later)
SSD: 2x 500GB samsung/crucial Sata Drives (raid 1 zfs A-SHIFT 13, 128GB passed to guest as VirtIO)
NVME: 2x 1TB Gigabyte Gen4 NVMe (raid 0 zfs A-SHIFT 13, 1.9TIB passed to guest as VirtIO)
Kernal: 6.2.6-1-pve #1 SMP PREEMPT_DYNAMIC PVE 6.2.6-1
PVE Manager Version: pve-manager/7.4-3/9002ab8a
root@maxwell:~# pveversion -v
proxmox-ve: 7.4-1 (running kernel: 6.2.6-1-pve)
pve-manager: 7.4-3 (running version: 7.4-3/9002ab8a)
pve-kernel-5.15: 7.4-1
pve-kernel-6.2: 7.3-8
pve-kernel-6.2.6-1-pve: 6.2.6-1
pve-kernel-5.15.104-1-pve: 5.15.104-1
pve-kernel-5.15.74-1-pve: 5.15.74-1
ceph-fuse: 15.2.17-pve1
corosync: 3.1.7-pve1
criu: 3.15-1+pve-1
glusterfs-client: 9.2-1
ifupdown2: 3.1.0-1+pmx3
ksm-control-daemon: 1.4-1
libjs-extjs: 7.0.0-1
libknet1: 1.24-pve2
libproxmox-acme-perl: 1.4.4
libproxmox-backup-qemu0: 1.3.1-1
libproxmox-rs-perl: 0.2.1
libpve-access-control: 7.4-2
libpve-apiclient-perl: 3.2-1
libpve-common-perl: 7.3-4
libpve-guest-common-perl: 4.2-4
libpve-http-server-perl: 4.2-1
libpve-rs-perl: 0.7.5
libpve-storage-perl: 7.4-2
libspice-server1: 0.14.3-2.1
lvm2: 2.03.11-2.1
lxc-pve: 5.0.2-2
lxcfs: 5.0.3-pve1
novnc-pve: 1.4.0-1
proxmox-backup-client: 2.4.1-1
proxmox-backup-file-restore: 2.4.1-1
proxmox-kernel-helper: 7.4-1
proxmox-mail-forward: 0.1.1-1
proxmox-mini-journalreader: 1.3-1
proxmox-widget-toolkit: 3.6.5
pve-cluster: 7.3-3
pve-container: 4.4-3
pve-docs: 7.4-2
pve-edk2-firmware: 3.20230228-1
pve-firewall: 4.3-1
pve-firmware: 3.6-4
pve-ha-manager: 3.6.0
pve-i18n: 2.12-1
pve-qemu-kvm: 7.2.0-8
pve-xtermjs: 4.16.0-1
qemu-server: 7.4-3
smartmontools: 7.2-pve3
spiceterm: 3.2-2
swtpm: 0.8.0~bpo11+3
vncterm: 1.7-1
zfsutils-linux: 2.1.9-pve1
Guest VM Settings
agent: 1
balloon: 0
bios: ovmf
boot: order=scsi1
cores: 12
cpu: host
efidisk0: local-zfs:vm-107-disk-0,efitype=4m,pre-enrolled-keys=1,size=1M
hostpci0: 0000:0a:00,pcie=1,x-vga=1
machine: pc-q35-7.2
memory: 24576
meta: creation-qemu=7.2.0,ctime=1680595875
name: Win10-Ligo
net0: virtio=4E:43:07:96:2B:CF,bridge=vmbr0,firewall=1,tag=50
numa: 1
ostype: win10
scsi0: Ligo:vm-107-disk-0,backup=0,discard=on,iothread=1,size=2000G,ssd=1
scsi1: local-zfs:vm-107-disk-1,cache=unsafe,discard=on,iothread=1,size=128G,ssd=1
scsihw: virtio-scsi-single
smbios1: uuid=e1907010-f732-4853-8565-7c534d9f69c8
sockets: 1
tags: vlan50-mgt
usb0: host=8087:0029
usb1: host=1e71:170e
usb2: host=046d:0ab1
usb3: host=046d:c21f
usb4: host=046d:c33c
usb5: host=046d:c08b
vmgenid: 61845dc8-df21-433e-90c3-d40ef255420b
BIOS options I have knowingly changed
Above 4G decoding: On
Resizable Bar: Off (causes VM to crash with no display after loading windows if ON)
xAPIC: set to x2APIC
VT-D/IOMMU: Enabled
CSM: Disabled
Things I have tried so far
Played around with Resizable bar and found it crashed the VM.
Orginally I had the host OS drive as an IDE, instead of VirtIO, changing it to virtio improved the frametimes but not that much.
Tried allocating all 24 cores to the guest VPM, no dice. Tried 4 cores, 6 cores, and now 12 cores. Doesn’t appear to change anything with lower amounts obviously slowing it down further.
Tried enabling all relevent AMD CPU flags in the proxmox menu, no change really.
Tried disabling balooning ram, no real improvement.
Tried buggering around with BIOS settings are PCIe, no real improvement.
Went from Kernal 5.15 to 6.2, no real improvement.
Enabled CPU NUMA on proxmox UI, unclear if it helped.
Send help my fellow magicians!
P.S, hope I made this nice and short for you…