Ryzen GPU Passthrough Setup Guide: Fedora 26 + Windows Gaming on Linux | Level One Techs

As anyone heard anymore improvements on virtualization from AMD, I am looking for any news about improvement on the bad performance the Zen chips have with KVM?

Kvm still bad if you don’t iommu io as well. Next tutorial will use xen if I can get it working.

1 Like

Thanks I was wondering if Xen would give better performance, keep up the good work.

I don’t know whats wrong with my setup. Windows 10 works fine but I can’t get any linux guest working.
I tried Ubuntu 16.04 LTS, Xubuntu 17.04 and Fedora 26 Workstation as guest.

No matter what I am trying, if I use GPU passthrough as I did for Windows 10 my guest and host system freezes on guest shutdown.
Someone here tried running a linux guest with GPU passthrough?

EDIT:
It looks like there is an issue with the “release” of the GPU on shutdown since using the linux guest works just fine, except for the shutdown.

Also does not seem to help:

Do not know why it seems impossible to run a linux guest on a linux host using gpu passthrough -.-

Frustration city. This project seems increasingly quixotic, for me at least. I can’t even get to first base. Changes to my grub config don’t take, my dracut doesn’t actually add the drivers, and basically vfio just doesn’t get loaded. I hope beyond hope that I am doing something very basic incorrectly and a simple “aha” moment will solve things. I follow the guide, I watch the videos, and it all comes to naught.

Is there anyone out there that has successfully accomplished this project with similar or identical hardware to mine?

  1. Asrock Taichi X370 (lastest BIOS as of Aug 26, IOMMU on, SRIOV on, virtualization on)
  2. Ryzen 7 1700X
  3. Brand new Fedora 26 install as of Aug 26.
  4. Primary GPU Radeon RX 550
  5. Secondary GPU GTX 1070

@wendell says that Intel is “boring”, but he also says that for the most part “it just works”. I am thinking I might have to move to boring+working from exciting+frustrating.

a lot of people on /r/VFIO are having more success with arch, not sure how much that helps.

Never had a problem doing the same on intel, even with mobos with broken IOMMU.

Guys, I know it’s all VFIO in here, but if you want to try things, try doing it with pci-stub??

  1. Add to Kernel Line in /etc/default/grub (instead of vfio)

amd_iommu=on pci-stub.ids=1002:6899,1002:aa50

  1. sudo grub2-mkconfig -o /boot/grub2/grub.cfg

Reboot

  1. To Check if pci-stub Worked

dmesg | grep pci-stub

  1. Try it

which part is not working? is the stub driver bound to your 1070?

I have a fedora 26 host and several VM including fedora 25 that I upgraded to 26. It work fine up until I think upgrade to kernel 4.12.5. Now the VM wont boot or install fresh fedora 26 with just KVM / QEMU no passthrough. So there is a problem that needs to be fixed for either the host or guest with fedora 26.

Maybe some else can confirm this as Im not a expert but I have rolled back the guest too kernel 4.11-* via a snapshot and it works but will update and lock up on boot.

It always has nouveau bound to the 1070 and amdgpu bound to the RX 550. I would have thought the *.pre in the grub would have nixed all that.

Right after boot if you do an lsmod does it show vfio?

Finally got it working myself:

It seems that Linux guests working only using the Chipset i440FX (at least for me), but as far as I read just Q35 supports PCIE passthrough and i440FX only PCI passthrough, so I am still a little bit confused.

BTW: If you want to use synergy to share keyboard and mouse, you have to use Xorg desktop instead of wayland

The last thing that is not working properly for me is the mentioned “Saving VM state with GPU passthrough” and of course performance could be better.

2 Likes

Thanks for the tip. Your suggestion for the grub changes worked just fine and the stub is grabbing the GPU as intended.

Still not sure why the vfio is such a PITA for me, though.

1 Like

Yeah, I don’t know, what are the advantages of vfio vs stub?

Mainly vfio let’s you set power state to low power. Supposedly you can also disable the root hub to let many cards be reinitialized but idk about that.

Maybe you can do low power with stub now too… Not sure really

1 Like

So how’s the performance with your setup? If I ever decide to pull the trigger, I would get something like your setup. is nvidia behaving in the vm?

I don’t know, but I have seen that my 5850 can reboot with the vm if I’m on windows 10 without issues, but it will freeze the system with windows 7. That’s with vfio. I don’t recall with pci-stub.

Has there been any answers from AMD regarding the NPT issue, and the fact that Threadripper is “broken” with regards to PCIe passthrough ( reddit link )?

[root@localhost ~]# dracut -f --kver `uname -r`
/etc/dracut.conf.d/vfio.conf: line 1: vfio_iommu_type1: command not found`

Thoughts? getting the same error as Chop91 in journalctl. And still seeing the nvida drivers:
lspci -vnn

21:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM206 [GeForce GTX 960] [10de:1401] (rev a1) (prog-if 00 [VGA controller])
Subsystem: eVga.com. Corp. Device [3842:3967]
Flags: bus master, fast devsel, latency 0, IRQ 66
Memory at f4000000 (32-bit, non-prefetchable) [size=16M]
Memory at e0000000 (64-bit, prefetchable) [size=256M]
Memory at f0000000 (64-bit, prefetchable) [size=32M]
I/O ports at d000 [size=128]
[virtual] Expansion ROM at f5000000 [disabled] [size=512K]
Capabilities: [60] Power Management version 3
Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
Capabilities: [78] Express Legacy Endpoint, MSI 00
Capabilities: [100] Virtual Channel
Capabilities: [258] L1 PM Substates
Capabilities: [128] Power Budgeting <?>
Capabilities: [420] Advanced Error Reporting
Capabilities: [600] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Kernel driver in use: nvidia
Kernel modules: nouveau, nvidia_drm, nvidia

system:
Ryzen 1700
ASUS B350-Plus – bios 0806
gtx 1030 (primary slot)
gtx 960 (secondary, attempting to vfio)

kernel:
4.12.9-300.fc26.x86_64

lsinitrd should show you what modules are installed in the initial ramdisk. What distro and kernel ? Try find /var |grep vfio and let us know the output of those commands ?