70% game performance in vfio vm compared to bare metal

Running a Asus PROART X570-CREATOR, 64Gb of ram, and a Ryzen 9 5950x passing an RTX 3090 on Proxmox. I’m trying to split up the abundant cpu resources amongst some other VM’s with the primary focus being gaming, while not dedicating the entire machine to Winderz.

I don’t have much time to test but I’m playing Shadow of the Tomb Raider and getting 144 bare metal, minimal hitching.
But in the VM, i’m getting closer to 100, and a ton of hitching, which is the real issue.

The vm is passing through the boot drive, a pcie nvme, and the gpu without issue. I started giving the system only 8 cores, and 16gb of ram, but tried again at all cores, and 18gb of ram, same result.

The GPU is pegged at 100% on bare metal but only like 40-70% in the VM. In both, there is “normal”/expected cpu usage.

I feel like I’m missing something, some trick, some turnable.

What am i missing?

probably missing a CPU instruction set. Guaranteed if you left the CPU as the default KVM64. also test with NUMA.

https://www.nicksherlock.com/2017/10/installing-macos-high-sierra-on-proxmox-5/

this page gives a lot of good examples on some PCIE passthrough and CPU config. It uses OSX as the guest but that is not relevant.

ahh, my bad. I forgot to mention the VM’s cpu config:

CPU: host, hidden=1, Flages=+AES or something

I don’t get the impression that it’s cpu bound, but idk.

More or less followed this guide, didn’t need to do the ROM File though:

so you are on an AMD rig with an NVIDIA GPU. your FLAGS line is VERY important to get correct. ALSO, did you do an UEFI install with VIRTIO drivers? or a more generic install?

I used the OVMF option stated here:

https://pve.proxmox.com/wiki/Pci_passthrough#GPU_OVMF_PCI_Passthrough_.28recommended.29

The flag was the selectable AES one.

The system is powered off right now or I’d post the entire config file. If this needs to wait until then, I understand. I don’t want to be annoying, lol.

Was hoping something might pop out otherwise.

what CPU type do you have selected?
what version of Proxmox are you on?

Yes, something did pop out, you do not have the correct instruction sets set up.

Thanks for spitballing with me,

latest version of Proxmox, 7.1
I have host selected, with AES enabled, and hidden=1.

so:

cpu: host,hidden=1,flags=+aes

Hmm, I’ll have to boot it and check, but I might have kvm-off enabled

KVM-off is only designed to trick some things into thinking it is not in a VM. you still very much have to get the CPU config correct.

That config line I listed is correct… right?

Now I’m doubting everything.

This guy was saying it was bad to disable it, I can try is with out later:

Hmmm, maybe i can check out a few of these options:

you need to hide VM architecture from the NVIDIA driver. the 3090 series actually may NOT need that though, differnt driver versions offer different results. NVIDIA tends to treat the very top end gaming products more like QUADRO products.

honestly the guide i posted about OSX is one of the better passthrough guides.

Yeah, I thought we were over the Nvidia error 43 too, but it’s still there, and extremely annoying

1 Like

balloon: 0
bios: ovmf
boot: order=ide2;hostpci0;virtio0;net0
cores: 32
cpu: host,flags=+aes
efidisk0: local-lvm:vm-100-disk-1,efitype=4m,pre-enrolled-keys=1,size=4M
hostpci0: 0000:01:00,pcie=1
hostpci1: 0000:3f:00,pcie=1,x-vga=1
hostpci2: 0000:3a:00,pcie=1
ide2: local:iso/virtio-win-0.1.208.iso,media=cdrom,size=543390K
machine: pc-q35-6.1
memory: 18432
meta: creation-qemu=6.1.0,ctime=1637529972
name: Windowz
net0: virtio=F2:93:79:AB:BD:19,bridge=vmbr0,firewall=1
numa: 1
ostype: win11
scsi7: /dev/disk/by-id/ata-SanDisk_SSD_PLUS_1000GB_191177443311,size=953872M
scsi8: /dev/disk/by-id/ata-SanDisk_SSD_PLUS_1000GB_191177453601,size=953872M
scsi9: /dev/disk/by-id/ata-ST16000NM001G-2KK103_ZL2KNRCB,size=14902G
scsihw: virtio-scsi-pci
smbios1: uuid=5c953a69-7cfb-42de-809f-3da01f3d70cc
sockets: 1
tpmstate0: local-lvm:vm-100-disk-2,size=4M,version=v2.0
usb0: host=7-1.1.1,usb3=1
usb1: host=7-1.4,usb3=1
usb2: host=8-4,usb3=1
usb3: host=7-1.1.2,usb3=1
usb4: host=7-1.2,usb3=1
vga: none
virtio0: local-lvm:vm-100-disk-0,size=32G
vmgenid: 98eb1066-580e-406e-9b37-6eb0376b304c

Well, I swapped gpu slots and it seems to be working as expected. Hmm, i’ll keep an eye on it.