I’ve been using PCI passthrough for my past builds as daily driver. I like the technology very much. I recently built (my first) an AMD PC. Here’s my experience.
CPU: AMD 7950X
MoBo: ASRock X670E Steel Legend (AGESA 1.0.0.7b)
RAM: Corsair DDR5 6000mhz CL40 2x2x32
GPU1: MSI RTX 4080 (in PCI 5.0 slot)
GPU2: Gigabyte GTX 1080 (in PCI 3.0 slot)
Other: 1 NVME, 2 SATA SSD, 2 SATA HDD
Host OS: Arch kernel 6.4 (GTX 1080)
Guest OS: Win 10 guest 1 (RTX 4080 passed through)
Win 10 guest 2 (Raphael iGPU pass through work-in-progress)
Initial setup:
BIOS: No over-clocking, IOMMU enabled, advanced error reporting enabled, ACS enabled, reBAR enabled, SR-IOV disabled, primary output external GPU.
VR connected to 4080, monitor connected to 4080, monitor connected to 1080, no monitor connected to iGPU.
Followed Archlinux OVMF guide, virtualization hidden.
Notes: binding vfio-pci through kernel parameter was unsuccessful, while binding it through /etc/modprobe.d/vfio.conf did the trick.
Graphic only output through 4080 during boot sequence.
Host would freeze during boot after initramfs and loading udev.
After disconnecting the monitor and the VR on 4080, the host would pause for a moment and resumes booting.
Connecting the monitors back to 4080, starting guest 1 produces black screen.
reBAR disabled:
host still freezes during boot, guest 1 black screen
passing qemu arg “-fw_cfg opt/ovmf/X-PciMmio64Mb,string=65536”
guest 1 shows “Guest has not initialized the display (yet)”
efifb:off, bind amdgpu through /etc/X11/xorg.conf.d/10-gpus.conf, supplying RTX 4080 vBIOS in the xml of hostdevice section
host outputs bios and bootloader through 4080 only.
host video output through 4080 would freeze during boot sequence, the video output would shift to 1080.
guest 1 pass through achieved.
iGPU passthrough (still troubleshooting)
added vesafb:off, nofb, pci=realloc, and updated /etc/X11/xorg.conf.d/10-gpus.conf for all 3 GPU
successfully bind vfio-pci to AMD Raphael iGPU and sound devices, iGPU and sound devices were in different IOMMU groups
IOMMU group passed to guest 2 without crashing host. Guest 2 Code 43. AMD driver installer could not properly identify the iGPU.
Could not find vBIOS for Raphael iGPU.
Tried to dump vBIOS using UBU from MoBo BIOS or amdfbflash from live host OS, both failed. The former would return an empty folder, while the latter would prompt error message saying adaptor not found.
Found a vBIOS dump on this forum through cat /sys/kernel/debug/dri/0/amdgpu_vbios > vbios.rom. Code name was misspelled as rafael, but device ID matched.
Other people claimed that it is in fact a valid vBIOS, but it still got me a black screen for guest 2.
Found many recent posts on reddit complaining trouble passing through AMD 7000 series Raphael iGPU.
Side note:
The MoBo has 3 USB 3.0 controllers and 8 ports. 1 connected to CPU while the other 2 connected to a chipset through a PCIe 4.0 with other PCI sockets.
Passing the controller directly connected to CPU will crash host.
The other two controllers will not crash host, but can only be passed together even though they are in different IOMMU groups.