Thank you for that! This was my starting point few years back, when I originally tried KVM with my 3900X, then 5900X and now my 7950X3D 
Confirming I run this setup on an ASRock TRX50 w/ a 7960X CPU and RTX 3090 and single GPU passthrough works in Ubuntu 22.04.
I had to comment out the VTconsole unbind/rebind line of the vfio-startup.sh and vfio-teardown.sh scripts, because of the issue with kernel 6.x.
Thank you for that! This was my starting point few years back, when I originally tried KVM with my 3900X, then 5900X and now my 7950X3D
Most welcome. How is the 7950X3D? As compared to the 3900X? Any things to consider when tuning the VM (e.g. cpu pinning, etc.)?
I made a reddit post recently, cause I managed to play PUBG (BattlEye) and shared my setup. You can read the details here.
Comments and suggestions, most welcome!
1 Like
I canāt complain. I am on Proxmox - running two latency depended gaming VMs for streaming is no problem. (3080ti, AMD 6600 RX). I must admit, the system usually isnāt pushing many pixels, because Steamdeck handhelds with 800p resolution are the receptors most of the time. The systems feel very snappy, I donāt have the feeling I have to do CPU pinning or thread isolation at all. But I must admit, I have no task which really pushes the CPU.
I could use some advice.
Iām trying to get passthrough to work with a W790 board and a 4090. Proxmox wouldnāt run on this system (hangs at āloading initial ramdiskā after install) and since this seems to be an issue hitting others as well with no common solution, I went with Ubuntu as the hypervisor.
Followed a bunch of other peopleās notes, wasted a ton of time before discovering that the grub boot options donāt work for vfio binding with kernel 6+, and so on. Where I am right now is this:
IOMMU enabled, working. Thatās the only thing I have in grub boot options.
Using driverctl to bind the GPU + its audio component + a dedicated USB card to vfio. (On Ubuntu you have to blacklist or uninstall the nvidia drivers for this, because trying to unbind the nvidia kernel module causes driverctl to hang hard enough that only a reboot helps - but nouveau can be unbound at will so Iām doing that.)
The VM is just a stock thing created with virt-manager, and the three PCIe devices (GPU, nvidia audio, USB) added.
At first Iād get nothing but blackness on the displayport outs of the card when running the VM, and a curious final entry in the VM log about receiving an unaligned iommu request. The USB card wasnāt working either which caused me to believe that this is more than just the GPU refusing to do its thing in a VM. Following Google to an obscure reddit post led me to - bear with me here - replacing OVMF_CODE.secboot.fd with the one taken from Proxmox. This was a last-ditch effort before giving up on Ubuntu, but it got the 4090 and the USB card working! Except now rebooting the VM causes Ubuntu to reboot as well (it might be a kernel panic, I havenāt really investigated much.)
So anyway, this is where I am right now. Any thoughts / advice before I move on to some other hypervisor?
I think, it might help you to revert your āOVMF_CODE.secboot.fdā file back and try to add a second VGA to your VM (a virtual/software one), so you can see if the problem is your 4090, or something else.
Lots of people need to adjust some settings in their XML file, to make VM load drivers for VGAs.
I am always using this to my VM:
...
<graphics type="spice" autoport="yes">
<listen type="address"/>
<image compression="off"/>
</graphics>
...
<video>
<model type="virtio" heads="1" primary="yes">
<acceleration accel3d="no"/>
</model>
<address type="pci" domain="0x0000" bus="0x00" slot="0x01" function="0x0"/>
</video>
...
Thanks! I was able to get it to work, it was a stupid mistake; I did not replace the corresponding _VARS file so understandably qemu got confused.
I then spent some time on USB as the dedicated card I passed through would stop working on the first guest reboot. Passing in the motherboardās USB controller without binding it to vfio works though.
Sorry for late reply, been too busy lately.
Fedora 39 (also worked on Fedora 35 but had bad support for iGPU)
R7 7800x3D
Vega 56 + (Vendor reset module was needed)
UEFI/BIOS version: Asrock B650M PG Riptide 2.02
BIOS options I needed:
Advanced ā PCI Config ā āAbove 4G Decodingā disabled
Advanced ā NBIO Common ā IOMMU enabled
Working passing through to Win10 VM.
Some other quick thoughts on this board: iGPU support is good. Tons of IOMMU groups, every device is seperate. I didnāt need to use the ACS patch in grub like I did on my B450M board from Gigabyte.
An updated tutorial I made for users to be able to play games initializing BattlEye anti-cheat. No Kernel recompile, just XML settings. Previesuly it was working for PUBG, and I have to user two configurations, and also install Hyper-V. I managed to have single config, and Hyper-V is not needed anymore. Also, I am able to play Escape From Tarkov, which was impossible before.
If you experience problems with BE games, please let me know which game is kicking you, so I can maybe test.
https://www.reddit.com/r/VFIO/comments/18p8hkf/how_to_play_pubg_with_battleeye_on_a_windows_vm/
I got ākicked by punkbusterā in battlefield 4. Is your configuration able to fix that?
Havenāt play BF4 for years. You can just try, and see. It wonāt ban you, just kick.
Yeah, I got kicked after 5 minutes or so. I donāt know what the anticheat app is looking for.
Also I canāt get 3DMark score certified. It may be lacking of high precision timer.
In your configuration, which portion is the most relevant?
I didnāt go step-by-step, rather, I copied parts of another user who said his works. Most of them were there before, but I copied the whole sections.
The sections I copied were:
<sysinfo type="smbios"> (I replaced values with mine)
<features>
<cpu><feature policy.../>
<clock>
Thanks. Will try.
I changed HPET from ānoā to āyesā
<timer name="hpet" present="yes"/>
I didnāt see any difference (games still play), however:
- 3DMark/AIDA64 still do not work
- I had a ~1sec freeze while testing, and after that my audio was de-synced. I had to restart ālibvirtd.serviceā to get it synced again.
That fixed BF4 kick. Thanks.
1 Like
Same here.
In the recent kernels (come with Ubuntu), vfio-pci module is not loaded at boot by default. All screens go black as soon as vfio-pci module is loaded. I can verify it grabs the correct gpu that is intended to be passed through, but it also destroy the framebuffer of all other gpus that should remain functioning.
PS: I have the sense that it might be gpu related. AMDās driver failed to hold on the devices/framebuffers when vfio-pci module is loaded. I may purchase a cheap GT710 as host gpu to test.
This started since Kernel 6.x and I havenāt been able to fix this. Many guides around, one specifically for Kernel 6.x, but never worked for me.
The workaround I am using is this:
Since I have two monitors, A main in Linux (DP1) and VM (DP2), and B secondary in Linux (HDMI), I have A switched off until see the login screen. Then I switch it on, and login. Everything works fine.
With my previous monitor, where Linux was connected in HDMI (instead of DP1), I had to remove the cable from the VGA until I boot up the VM and it was very annoying.
Had the same problem with ADM 6900XT and RTX4080, so it is not AMD related.
I mean it may be related to the gpu used by the host, not for the guest.
You are using Ryzen igpu as the host gpu, right?
I remember since I got this issue, I switched to AMD R7 240 card for the host. It did resolve the issue. However, the issue emerged again after couple kernel update afterwards. Next, I will try nvidia gpu as the host gpu.