Ubuntu 17.04 -- VFIO PCIe Passthrough & Kernel Update (4.14-rc1)

Going dark for the rest of today…my HTPC just hit me with a

“failed to isolate default target freezing”

Have absolutely no clue that this is but it happened after I had to fsck my boot device. So I may not answer questions today

Oh, good points - thanks!

It looks like i can’t use it for gaming, if i understand correctly? :frowning: Sad news.

Do you agree?

See this issue, it has solution that has worked for a couple people.

1 Like

OK, so I tried this with both VGA devices (08:00 and 09:00) and neither worked. Both times the display just froze. So I don’t think it is differentiating between the two different cards, even though they’re in separate IOMMU groups.

Here is my lspci output:

08:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480] [1002:67df] (rev e7) (prog-if 00 [VGA controller])
        Subsystem: Sapphire Technology Limited Ellesmere [Radeon RX 470/480/570/580] (Nitro+ Radeon RX 580 4GB) [1da2:e366]
        Flags: bus master, fast devsel, latency 0, IRQ 75
        Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Memory at f0000000 (64-bit, prefetchable) [size=2M]
        I/O ports at d000 [size=256]
        Memory at fe900000 (32-bit, non-prefetchable) [size=256K]
        Expansion ROM at 000c0000 [disabled] [size=128K]
        Capabilities: <access denied>
        Kernel driver in use: amdgpu
        Kernel modules: amdgpu

08:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [1002:aaf0]
        Subsystem: Sapphire Technology Limited Ellesmere [Radeon RX 580] [1da2:aaf0]
        Flags: bus master, fast devsel, latency 0, IRQ 82
        Memory at fe960000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
09:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480] [1002:67df] (rev e7) (prog-if 00 [VGA controller])
        Subsystem: Sapphire Technology Limited Ellesmere [Radeon RX 470/480/570/580] (Nitro+ Radeon RX 580 4GB) [1da2:e366]
        Flags: bus master, fast devsel, latency 0, IRQ 77
        Memory at c0000000 (64-bit, prefetchable) [size=256M]
        Memory at d0000000 (64-bit, prefetchable) [size=2M]
        I/O ports at c000 [size=256]
        Memory at fe800000 (32-bit, non-prefetchable) [size=256K]
        Expansion ROM at fe840000 [disabled] [size=128K]
        Capabilities: <access denied>
        Kernel driver in use: amdgpu
        Kernel modules: amdgpu

09:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [1002:aaf0]
        Subsystem: Sapphire Technology Limited Ellesmere [Radeon RX 580] [1da2:aaf0]
        Flags: bus master, fast devsel, latency 0, IRQ 84
        Memory at fe860000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: <access denied>
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel

And I tried running both sudo bash vfio-pci-bind.sh 0000:08:00.0 0000:08:00.1 and sudo bash vfio-pci-bind.sh 0000:09:00.0 0000:09:00.1, and both times the screen froze (but audio continued to work).

EDIT: Running it with sh instead gave me this output:

/usr/local/sbin/vfio-pci-bind.sh: 53: /usr/local/sbin/vfio-pci-bind.sh: [[: not found
/usr/local/sbin/vfio-pci-bind.sh: 58: /usr/local/sbin/vfio-pci-bind.sh: [[: not found
/usr/local/sbin/vfio-pci-bind.sh: 60: /usr/local/sbin/vfio-pci-bind.sh: [[: not found
Please supply Domain:Bus:Device.Function of PCI device in form: dddd:bb:dd.f

I did notice, while rebooting my PC, that I’m getting an error message when Linux starts saying:

sh: /usr/bin/vfio-pci-override.sh: not found

Well, /usr/bin/vfio-pci-override.sh does exist, and has these contents:

 #!/bin/sh    
for i in /sys/bus/pci/devices/*/boot_vga; do
            if [ $(cat "$i") -eq 0 ]; then
                    GPU="${i%/boot_vga}"
                    AUDIO="$(echo "$GPU" | sed -e "s/0$/1/")"
                    echo "vfio-pci" > "$GPU/driver_override"
                    if [ -d "$AUDIO" ]; then
                            echo "vfio-pci" > "$AUDIO/driver_override"
                    fi
            fi
    done 

I checked the permissions on the file: -rwxr-xr-x 1 root root 286

So now I’m getting pretty confused. I found this Arch thread from yesterday that seems to confirm I don’t need to bother with the initcpio hook:

https://bbs.archlinux.org/viewtopic.php?id=242847

So I don’t think it’s that.

I may just grab the Vega 64 Newegg has on sale right now, so I don’t have to deal with this same-gpu nonsense. I really just want to get Star Citizen running in Linux somehow…

Hi, Happy New Year.

Sorry for the late reply. Been dark for Christmas. Yeah, the identical card stuff is a real pain in the behind. Did you get the Vega?

While you may be waiting for the Vega, you can try installing your preferred Windows OS and Star Citizen, Alpha 3.4.1 in a VirtualBox VM or even a standard Virt-manager windows VM without GPU passthrough.

Be sure you pass an external hard drive to the VM or share a folder location on your Linux home with the VM.

Once you have installed Alpha 3.4.1 copy the “Star Citizen” folder to the external hard drive or shared folder

Now go to the Lutris website download and install Lutris.

Next go to the Star Citizen page on the Lutris website

https://lutris.net/games/star-citizen/

Click install on the line with the Live Version

This download and install the required Version of Wine and all dependencies of the game. It will then install the launcher.

Once the launcher is installed, you can open the game but don’t log in just yet. Instead go to the folder you copied the Star Citizen folder from the VM into and copy the entire folder into the "~/program files/robert space industries/ folder Lutris created.
Mine looks like this
30
You have to do the Windows install and copy till we can get Wine to emulate the Windows admin account privileges. It is getting there but not completely. Before the launcher would only download 13GB of the Data.p4k file, now it is downloading all but 10MB or so of the ~49GB. Hopefully, Wine 4.20 will fix this completely.

Anyway, once you’ve copied the folder over to your Lutris star-citizen prefix, you can log in and allow the launcher to install the game. If everything works as it should, the launcher should just flash from saying “Install” to saying “Launch Game”
Assuming you have Vulkan installed,

$ sudo add-apt-repository ppa:oibaf/graphics-drivers
$ sudo apt update
$ sudo apt upgrade
$ sudo apt install libvulkan1 mesa-vulkan-drivers vulkan-utils

The game will launch and you can play the game. FOIP will not work and will crash the game if you try to enable it last I checked in 3.3.7.
My main issue with 3.3.7 was that it would through a .Net error and crash after about 10mins. I have not had this issue in 3.4.1 though I have only tested it 2 times and not in serious game play, more just opening the PU stepping outside the room and leaving the game alone.
I will do more testing later this week maybe even during the day later today to verify. But my guesses are that the .Net error bug has been fixed by CIG though there are some memory leakage bugs still in the game that Linux/Wine is identifying more so than Windows.

See this thread on the LUX Spectrum forum.

https://robertsspaceindustries.com/spectrum/community/LUG/forum/149/thread/its-working-open-mouth-full-persistent-universe/1842813

My guesses are that once we get to March and Alpha 3.5, there may be a switch from DX11 to Vulkan which may really improve Linux compatibility. All that said, the main issues are in the persistent universe, both Arena Commander and Star Marine have been working fine since Alpha 3.2 according to the posts on Spectrum.

1 Like

I really want to get this working. My hopeful setup:

  • Ubuntu 18.10 on Intel 6700k Integrated Graphics
  • GTX 980ti passthrough to Windows 10 VM

lspci output:

00:02.0 VGA compatible controller [0300]: Intel Corporation HD Graphics 530 [8086:1912] (rev 06) (prog-if 00 [VGA controller])
	Subsystem: Micro-Star International Co., Ltd. [MSI] HD Graphics 530 [1462:7977]
	Flags: bus master, fast devsel, latency 0, IRQ 140
	Memory at dd000000 (64-bit, non-prefetchable) [size=16M]
	Memory at b0000000 (64-bit, prefetchable) [size=256M]
	I/O ports at f000 [size=64]
	[virtual] Expansion ROM at 000c0000 [disabled] [size=128K]
	Capabilities: <access denied>
	Kernel driver in use: i915
	Kernel modules: i915

02:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce GTX 980 Ti] [10de:17c8] (rev a1) (prog-if 00 [VGA controller])
	Subsystem: eVga.com. Corp. GM200 [GeForce GTX 980 Ti] [3842:4996]
	Flags: bus master, fast devsel, latency 0, IRQ 141
	Memory at de000000 (32-bit, non-prefetchable) [size=16M]
	Memory at c0000000 (64-bit, prefetchable) [size=256M]
	Memory at d0000000 (64-bit, prefetchable) [size=32M]
	I/O ports at e000 [size=128]
	Expansion ROM at df000000 [disabled] [size=512K]
	Capabilities: <access denied>
	Kernel driver in use: nouveau
	Kernel modules: nvidiafb, nouveau

Right now I’m stuck at the “getting vfio working” step. Here’s what I’ve done:

initramfs-tools/modules

softdep nouveau pre: vfio vfio_pci
softdep nvidiafb pre: vfio vfio_pci
softdep i915 pre: vfio vfio_pci # tried with this enabled and commented out

vfio
vfio_iommu_type1
vfio_virqfd
options vfio_pci ids=8086:1912,10de:17c8
vfio_pci # doesn't seem to like the ids here
i915 # tried with this enabled and commented out
nouveau

/etc/modules

vfio
vfio_iommu_type1
vfio_pci # doesn't seem to like the ids here

modprobe.d/nouveau.conf

softdep nouveau pre: vfio vfio_pci

modprobe.d/vfio_pci.conf

options vfio_pci ids=8086:1912,10de:17c8

I’m using the 4.16.18 kernel, because it has the Nvidia and Intel firmware.


I also edited the grub file to enable IOMMU, and ensured it’s enabled in the BIOS settings. If I edit the boot arguments within grub on startup and remove the IOMMU enables, the system will boot up. But, if I don’t disable IOMMU the system will hang either at a blank black screen, a blank “ubuntu orange” screen, or a blank black screen with a single solid cursor at the top right. It depends on what configuration the different stuff is in.

Here’s some abbreviated journalctl logs that might be helpful:

Jan 08 00:03:51 desktop kernel: vfio-pci 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem
Jan 08 00:03:51 desktop kernel: vfio_pci: add [8086:1912[ffff:ffff]] class 0x000000/00000000
Jan 08 00:03:51 desktop kernel: vfio-pci 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Jan 08 00:03:51 desktop kernel: vfio_pci: add [10de:17c8[ffff:ffff]] class 0x000000/00000000

Jan 08 00:04:05 desktop gnome-shell[1474]: Failed to create backend: Could not find a primary drm kms device
Jan 08 00:04:05 desktop gnome-session[1460]: gnome-session-binary[1460]: WARNING: App 'org.gnome.Shell.desktop' exited with code 1
Jan 08 00:04:05 desktop gnome-session-binary[1460]: WARNING: App 'org.gnome.Shell.desktop' exited with code 1
Jan 08 00:04:05 desktop gnome-session-binary[1460]: Unrecoverable failure in required component org.gnome.Shell.desktop
Jan 08 00:04:05 desktop gdm-launch-environment][1383]: pam_unix(gdm-launch-environment:session): session closed for user gdm
Jan 08 00:04:05 desktop gdm3[1372]: GdmDisplay: display lasted 0.430835 seconds
Jan 08 00:04:05 desktop systemd-logind[915]: Session c1 logged out. Waiting for processes to exit.
Jan 08 00:04:05 desktop systemd-logind[915]: Removed session c1.
Jan 08 00:04:05 desktop gdm3[1372]: Child process -1451 was already dead.
Jan 08 00:04:05 desktop systemd[1]: Received SIGRTMIN+21 from PID 403 (plymouthd).
Jan 08 00:04:05 desktop gdm3[1372]: Child process -1451 was already dead.
Jan 08 00:04:05 desktop systemd[1]: Started Hold until boot process finishes up.

Jan 08 00:04:05 desktop kernel: vfio-pci 0000:02:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
Jan 08 00:04:05 desktop kernel: vfio-pci 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=io+mem

It appears that gnome is failing to load and this results in the system shutting down, but the computer remaining powered on.

In the case where IOMMU is disabled, I get the following:

Jan 08 00:05:36 desktop kernel: vfio-pci: probe of 0000:02:00.0 failed with error -22
Jan 08 00:05:36 desktop kernel: vfio_pci: add [10de:17c8[ffff:ffff]] class 0x000000/00000000

Which is the error code you seem to get when IOMMU is unavailable.


I am running sudo update-initramfs -u -k 4.16.18-041618-generic and sudo update-grub after making any changes.

I hope someone can help me. Thanks.

Edit:

Finally figured out my problem. I thought you were supposed to use the addresses for both GPUs. You should have the addresses for the GPU and Audio controller only of the device you wish to passthrough.

Hi @wendell and #everyone , I’ve just finished setting up a VM - launch it and get into a EFI grub menu rather than windows, I’m assuming I’m doing something incorrectly with the drive:

This is the output:

UEFI Interactive Shell v2.2
EDK II
UEFI v2.70 (EDK II, 0x00010000)
Mapping table
     BLK0: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x0)
     BLK1: Alias(s):
          PciRoot(0x0)/Pci(0x1,0x0)/Floppy(0x1)
     BLK2: Alias(s):
          PciRoot(0x0)/Pci(0x9,0x0)/Sata(0x0,0xFFFF,0x0)
     BLK3: Alias(s):
          PciRoot(0x0)/Pci(0x9,0x0)/Sata(0x0,0xFFFF,0x0)/HD(1,MBR,0x073C329B,0x800,0xFA000)
     BLK4: Alias(s):
          PciRoot(0x0)/Pci(0x9,0x0)/Sata(0x0,0xFFFF,0x0)/HD(2,MBR,0x073C329B,0xFA800,0x19000000)
     BLK5: Alias(s):
          PciRoot(0x0)/Pci(0x9,0x0)/Sata(0x0,0xFFFF,0x0)/HD(3,MBR,0x073C329B,0x190FAFC1,0x2119A83F)
     BLK7: Alias(s):
          PciRoot(0x0)/Pci(0x9,0x0)/Sata(0x0,0xFFFF,0x0)/HD(4,MBR,0x073C329B,0x3A296000,0xEF000)
     BLK6: Alias(s):
          PciRoot(0x0)/Pci(0x9,0x0)/Sata(0x0,0xFFFF,0x0)/HD(3,MBR,0x073C329B,0x190FAFC1,0x2119A83F)/HD(1,MBR,0x00000000,0x190FB000,0x2119A800)
Shell> SC in 1 seconds to skip startup.nsh or any other key to continue.
Shell>

I’m currently trying to set the VM to boot from a SSD I’ve got plugged into sata port x on the mb. Linux sees it as /dev/sdd and it detects there is a Windows install on that device. My VM config for the drive looks like this:

<disk type='block' device='disk'>
      <driver name='qemu' type='raw' cache='none' io='native'/>
      <source dev='/dev/sdd'/>
      <target dev='hda' bus='sata'/>
      <address type='drive' controller='0' bus='0' target='0' unit='0'/>
    </disk>

Device info 
Device     Boot     Start       End   Sectors   Size Id Type
/dev/sdd1  *         2048   1026047   1024000   500M  7 HPFS/NTFS/exFAT
/dev/sdd2         1026048 420456447 419430400   200G  7 HPFS/NTFS/exFAT
/dev/sdd3       420458433 975788031 555329599 264.8G  f W95 Ext'd (LBA)
/dev/sdd4       975790080 976769023    978944   478M 27 Hidden NTFS WinRE
/dev/sdd5       420458496 975788031 555329536 264.8G  7 HPFS/NTFS/exFAT

I would like to passthrough a nvme but the problem is it’s hard to determine what are my NVMe’s IOMMU/PCI tags as they are WD Black and don’t appear correctly or rather there is no mention of them in the lspci list/script list when grepping.

Here is the output from the IOMMU list script:

IOMMU Group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 10 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU Group 11 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 59)
IOMMU Group 11 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 12 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1460]
IOMMU Group 12 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1461]
IOMMU Group 12 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1462]
IOMMU Group 12 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1463]
IOMMU Group 12 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1464]
IOMMU Group 12 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1465]
IOMMU Group 12 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric Device 18h Function 6 [1022:1466]
IOMMU Group 12 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1022:1467]
IOMMU Group 13 00:19.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1460]
IOMMU Group 13 00:19.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1461]
IOMMU Group 13 00:19.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1462]
IOMMU Group 13 00:19.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1463]
IOMMU Group 13 00:19.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1464]
IOMMU Group 13 00:19.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1465]
IOMMU Group 13 00:19.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric Device 18h Function 6 [1022:1466]
IOMMU Group 13 00:19.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1022:1467]
IOMMU Group 14 01:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43ba] (rev 02)
IOMMU Group 14 01:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b6] (rev 02)
IOMMU Group 14 01:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b1] (rev 02)
IOMMU Group 14 02:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 14 02:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 14 02:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 14 02:06.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 14 02:07.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 14 04:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU Group 14 05:00.0 Network controller [0280]: Intel Corporation Device [8086:24fb] (rev 10)
IOMMU Group 14 06:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU Group 15 08:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Device [15b7:5002]
IOMMU Group 16 09:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1)
IOMMU Group 16 09:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
IOMMU Group 17 0a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device [1022:145a]
IOMMU Group 18 0a:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor [1022:1456]
IOMMU Group 19 0a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller [1022:145c]
IOMMU Group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 20 0b:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device [1022:1455]
IOMMU Group 21 0b:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU Group 22 0b:00.3 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller [1022:1457]
IOMMU Group 23 40:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 24 40:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 25 40:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 26 40:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 27 40:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 28 40:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 29 40:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 2 00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 30 40:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU Group 31 40:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 32 40:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU Group 33 41:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Device [15b7:5002]
IOMMU Group 34 42:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480] [1002:67df] (rev c7)
IOMMU Group 34 42:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [1002:aaf0]
IOMMU Group 35 43:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device [1022:145a]
IOMMU Group 36 43:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor [1022:1456]
IOMMU Group 37 43:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller [1022:145c]
IOMMU Group 38 44:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device [1022:1455]
IOMMU Group 39 44:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU Group 3 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 4 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 5 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 6 00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 7 00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 8 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU Group 9 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]

I’m getting successfull output from the GPU on a secondary monitor when launching the vm - which leads me to believe that that is fine.

Specs listed below:

Ryzen 1920x - Asrock Taichi x399 (full size)
8 GB Ram (I’m getting more just using this to setup)
RX 480 8 GB (host)
GTX 1070 (passhtrough) - *I’ve done the code 43 workaround
2 x WD Black 500 GB NVMe drives running linux off of one want the other for windows
2 x 4 TB Drives (Raid 1) - irrelevant
PSU - irrelevant

I have no other devices connected to the MB.

Apologies for the long post - first time poster here !

If you need config files etc. please ask.

Hi all,

I was wondering if anyone could offer some advice - I’m currently up to “We downloaded the Windows ISO from Microsoft and set it up using this method.”

The problem is that the virtual machine has a black screen. I’m not sure where I’ve gone wrong as I can clearly see that Kernel driver in use: vfio-pciis set correctly for both devices. I’m not sure what logs I should be looking at in /var/lognor what I should grep.

Any advice on how to troubleshoot this would be appreciated.

  • Ubuntu 18.04, Dell XPS 13 with Intel iGPU
  • Radeon RX 580 passthrough to Windows 10 VM

I am trying to setup a raw disk as my boot device. I used these settings in the xml file.

<disk type='block' device='disk'>
  <driver name='qemu' type='raw' cache='none'/>
  <source dev='/dev/disk/by-id/nvme-Samsung_SSD_970_EVO_1TB_S467NX0KC27998E'/>
  <target dev='vda' bus='sata'/>
  <boot order='1'/>
  <address type='drive' controller='0' bus='0' target='0' unit='0'/>
</disk>

The raw disk is an nvme drive. It boots to grub.

And ideas how to make this work?

EDIT
I think I’ve discovered what is wrong. My mbr is on the nvme drive and grub is loaded on a ssd drive so when I pass through the nvme grub is no where to be found. I am not sure the best way to fix this so I’ll post a question in the linux software forum.