RX480 problems on KVM

Operating Systems & Open Source Linux
1

I have a working KVM but when I add the RX480 I am running into issues. It seems to work until I try install the drivers or Windows tries to do it automatically.

Machine;
i7-6700k
32gb ram
AMD 7870 primary card for the host Fedora 26 (in first PCI slot)
RX480 for the Windows 10 machine. (in third PCI slot)
ASRock Z170M Extreme4 LGA 1151 Intel Z170

I was running this as an unRAID KVM for 6-7 months with no issues passing the RX480 to Windows 10. I moved to Fedora because Steam VAC blocks playing CSGO on a VM. So I added the old 7870 video card so I can play from the host OS (handles CSGO fine). So I know the RX480 can work.

I spent last night messing with it and finally got PCI passthrough to display on my screen via the RX480. The problem is I can’t get the drivers fully installed before it reboots into recovery. I can switch back to spice and login to Windows then delete the video/audio RX480 drivers from device manager (show all drivers option). After that I can get back in to Windows via the RX480 but still fail to fully install all of the drivers/install I download from AMD. It seems like windows is installing the drivers it download from somewhere that doesn’t work. I have removed the network cable so it can’t download them but who knows if it saved them somewhere. I am not sure how to get it going.

after it fails here is the error.
From journalctl -u libvirtd.service --no-pager --follow
Aug 29 17:08:41 Blair2017-Linux libvirtd[989]: 2017-08-29 22:08:41.124+0000: 989: error : qemuMonitorIO:697 : internal error: End of file from qemu monitor

2

are you using the ACS patch? IOMMU is just plain broken on a lot of z170 boards

3

no, how do I do that???

4

This ? https://level1techs.com/video/ryzen-iommu-pcie-passthrough-works

5

gotta install or compile a patched kernel and add the pcie_acs_override=downstream flag to your bootloader parameters

https://wiki.archlinux.org/index.php/PCI_passthrough_via_OVMF#Bypassing_the_IOMMU_groups_.28ACS_override_patch.29

note that it could also be the non-isolated cards misbehaving and trying to operate in crossfire depending on your non-patched IOMMU groupings

if driver insallation and updates are your problems you may need to temporarily set the CPU type to core2duo in your configs, update and install, then change it back.

6

How would one do such things… Linux still painful after 15 years…

7

I should note that the RX480 is in it’s own group. Of course the PCI USB isn’t, that was the next problem on the list but currently I am not passing it through.

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers [8086:191f] (rev 07)
IOMMU Group 10 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8] (rev 31)
IOMMU Group 11 04:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1142 USB 3.1 Host Controller [1b21:1242]
**IOMMU Group 12 05:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/580] [1002:67df] (rev c7)**
**IOMMU Group 12 05:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:aaf0]**
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
IOMMU Group 1 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) [8086:1905] (rev 07)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Pitcairn XT [Radeon HD 7870 GHz Edition] [1002:6818]
IOMMU Group 1 01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cape Verde/Pitcairn HDMI Audio [Radeon HD 7700/7800 Series] [1002:aab0]
IOMMU Group 1 02:00.0 USB controller [0c03]: Renesas Technology Corp. uPD720201 USB 3.0 Host Controller [1912:0014] (rev 03)
IOMMU Group 2 00:14.0 USB controller [0c03]: Intel Corporation Sunrise Point-H USB 3.0 xHCI Controller [8086:a12f] (rev 31)
IOMMU Group 2 00:14.2 Signal processing controller [1180]: Intel Corporation Sunrise Point-H Thermal subsystem [8086:a131] (rev 31)
IOMMU Group 3 00:16.0 Communication controller [0780]: Intel Corporation Sunrise Point-H CSME HECI #1 [8086:a13a] (rev 31)
IOMMU Group 4 00:17.0 SATA controller [0106]: Intel Corporation Sunrise Point-H SATA controller [AHCI mode] [8086:a102] (rev 31)
IOMMU Group 5 00:1b.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Root Port #17 [8086:a167] (rev f1)
IOMMU Group 6 00:1c.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #1 [8086:a110] (rev f1)
IOMMU Group 7 00:1c.4 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #5 [8086:a114] (rev f1)
IOMMU Group 8 00:1d.0 PCI bridge [0604]: Intel Corporation Sunrise Point-H PCI Express Root Port #9 [8086:a118] (rev f1)
IOMMU Group 9 00:1f.0 ISA bridge [0601]: Intel Corporation Sunrise Point-H LPC Controller [8086:a145] (rev 31)
IOMMU Group 9 00:1f.2 Memory controller [0580]: Intel Corporation Sunrise Point-H PMC [8086:a121] (rev 31)
IOMMU Group 9 00:1f.3 Audio device [0403]: Intel Corporation Sunrise Point-H HD Audio [8086:a170] (rev 31)
IOMMU Group 9 00:1f.4 SMBus [0c05]: Intel Corporation Sunrise Point-H SMBus [8086:a123] (rev 31)
8

it looks like your cards are already isolated. Try the C2D update trick. Are you passing through both the AMD audio controller and the VGA device?

9

Trying the core2duo now.

I am passing both through. I have tired removing audio but it doesn’t work either. I seem to recall when I was first playing with this computer (end of last year) with proxmox that the HDMI causes some issues. I am using a 4k tv.

10

you want to be passing both through. Just going over the boilerplate.

11

c2d change didn’t seem to help

13

FUCK LINUX it has ALWAYS SUCKED

14

sounds like you’re just mad at it.

have you tried asking the /r/VFIO discord yet

15

yeah, mad. I am ready toss my computer a cross the room to get linux out of it.

I installed my first linux build in like 2000. It was the same story. hours of messing with it just get crap that was supposed to be working to work. So painful…

16

lol just use windows 10 then damn

17

I am kind of amp’ed up about virtualization. I have wanted to do this since 2010 . Of course using VMware. The tech is finally here and I needed to build a new box. I’d much rather use something else but my day job (networking industry) is moving towards virt’ing everything with opensuck software. So learning KVM has value attached. I check in with linux ever few years hoping it is better and it never is.

Looks like my answer is right here. In case you end up on this page from a google search for rebooting or crashing during video driver installation.
https://wiki.lime-technology.com/UnRAID_Manual_6#Creating_Your_Own_Virtual_Machines
The Machine type presented to your VM can be toggled between QEMU’s i440fx or Q35 chipsets

** For Windows-based VMs, i440fx is the default setting and should only be changed if you are having difficulty passing through a PCI-based graphics card (this may prompt Windows to reactivate)**
** For Linux-based VMs, Q35 is the default setting and should not be changed if passing through a GPU**

18

Then RTFM instead of screaming into the void about it lol

I was actually gonna suggest chipset next funnily enough

19

I tired to read the manual, Looks like they fucked it up when they went to Fedora 26, it is no more… https://docs.fedoraproject.org/index.html
what looks like helpful google links go to useless pages now. :frowning:

Next problem… The USB PCI card that is a shared group…

error
Error starting domain: internal error: qemu unexpectedly closed the monitor: 2017-08-30T18:55:28.388858Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/2 (label charserial0)
2017-08-30T18:55:29.552196Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,id=hostdev2,bus=pci.0,addr=0xa: vfio error: 0000:02:00.0: group 1 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

Traceback (most recent call last):
File “/usr/share/virt-manager/virtManager/asyncjob.py”, line 89, in cb_wrapper
callback(asyncjob, *args, **kwargs)
File “/usr/share/virt-manager/virtManager/asyncjob.py”, line 125, in tmpcb
callback(*args, **kwargs)
File “/usr/share/virt-manager/virtManager/libvirtobject.py”, line 82, in newfn
ret = fn(self, *args, **kwargs)
File “/usr/share/virt-manager/virtManager/domain.py”, line 1489, in startup
self._backend.create()
File “/usr/lib64/python2.7/site-packages/libvirt.py”, line 1039, in create
if ret == -1: raise libvirtError (‘virDomainCreate() failed’, dom=self)
libvirtError: internal error: qemu unexpectedly closed the monitor: 2017-08-30T18:55:28.388858Z qemu-system-x86_64: -chardev pty,id=charserial0: char device redirected to /dev/pts/2 (label charserial0)
2017-08-30T18:55:29.552196Z qemu-system-x86_64: -device vfio-pci,host=02:00.0,id=hostdev2,bus=pci.0,addr=0xa: vfio error: 0000:02:00.0: group 1 is not viable
Please ensure all devices within the iommu_group are bound to their vfio bus driver.

You’ll get a kick out of this… same problem I was having…

20

for real dawg try the /r/VFIO discord. they’re an incredibly helpful bunch

21

I did start to have some issues when I fired up a game. I had to do the C2D trick to get the drivers installed.

1 Like