Iommu. GPU Passthrough problem because of identical GPU’s

Hello!

since few weeks, i try to switch (back) to Linux as Desktopsystem. I use Linux as Serversystem since 15 Years (Redhat, Debian). And as Desktop i used Linux from 2006 - 2013.

2013 i started to play Videogames, and so i switched back to Windows.

Now i switched to Manjaro (because last “Linuxdesktopsession” i used Arch Linux). And got few Problems with my nvidia GPU’s, so i buyed a thrid GPU from AMD. A Radeon Pro WX3100.

Here are the Specs from the System:

System:
  Host: colossus Kernel: 5.1.0-mainline x86_64 bits: 64 compiler: gcc 
  v: 8.3.0 Console: tty 0 Distro: Manjaro Linux 
Machine:
  Type: Desktop Mobo: ASUSTeK model: ROG ZENITH EXTREME v: Rev 1.xx 
  serial: <filter> UEFI: American Megatrends v: 1701 date: 01/09/2019 
CPU:
  Topology: 16-Core (2-Die) model: AMD Ryzen Threadripper 1950X bits: 64 
  type: MT MCP MCM arch: Zen rev: 1 L2 cache: 8192 KiB 
  flags: lm nx pae sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm 
  bogomips: 256070 
  Speed: 2000 MHz min/max: 2200/4000 MHz Core speeds (MHz): 1: 3120 2: 2000 
  3: 2200 4: 2200 5: 2000 6: 2000 7: 2000 8: 3999 9: 3223 10: 2000 11: 3999 
  12: 2000 13: 3999 14: 2200 15: 2200 16: 2200 17: 2200 18: 2200 19: 2200 
  20: 2200 21: 2200 22: 2000 23: 2000 24: 2000 25: 2000 26: 3999 27: 2200 
  28: 2200 29: 2200 30: 2200 31: 2200 32: 2200 
Graphics:
  Device-1: NVIDIA GP102 [GeForce GTX 1080 Ti] vendor: ASUSTeK 
  driver: nouveau v: kernel bus ID: 08:00.0 
  Device-2: AMD Lexa XT [Radeon PRO WX 3100] driver: amdgpu v: kernel 
  bus ID: 42:00.0 
  Device-3: NVIDIA GP102 [GeForce GTX 1080 Ti] vendor: ASUSTeK 
  driver: nouveau v: kernel bus ID: 43:00.0 
  Display: server: X.org 1.20.4 driver: nouveau 
  resolution: <xdpyinfo missing> 
  OpenGL: renderer: AMD Radeon Pro WX3100 (POLARIS12 DRM 3.30.0 
  5.1.0-mainline LLVM 8.0.0) 
  v: 4.5 Mesa 19.0.3 direct render: Yes 
Audio:
  Device-1: NVIDIA GP102 HDMI Audio vendor: ASUSTeK driver: snd_hda_intel 
  v: kernel bus ID: 08:00.1 
  Device-2: AMD Family 17h HD Audio vendor: ASUSTeK driver: snd_hda_intel 
  v: kernel bus ID: 0a:00.3 
  Device-3: AMD Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] 
  driver: snd_hda_intel v: kernel bus ID: 42:00.1 
  Device-4: NVIDIA GP102 HDMI Audio vendor: ASUSTeK driver: snd_hda_intel 
  v: kernel bus ID: 43:00.1 
  Sound Server: ALSA v: k5.1.0-mainline 
Network:
  Device-1: Intel I211 Gigabit Network vendor: ASUSTeK driver: igb 
  v: 5.6.0-k port: 2000 bus ID: 03:00.0 
  IF: enp3s0 state: up speed: 1000 Mbps duplex: full mac: <filter> 
  Device-2: Aquantia AQC107 NBase-T/IEEE 802.3bz Ethernet [AQtion] 
  vendor: ASUSTeK driver: atlantic v: 2.0.4.0-kern port: 2000 
  bus ID: 05:00.0 
  IF: enp5s0 state: up speed: 10000 Mbps duplex: full mac: <filter> 
Drives:
  Local Storage: total: 2.73 TiB used: 85.24 GiB (3.1%) 
  ID-1: /dev/nvme0n1 vendor: Samsung model: SSD 970 EVO 1TB size: 931.51 GiB 
  ID-2: /dev/nvme1n1 vendor: Samsung model: SSD 970 EVO 1TB size: 931.51 GiB 
  ID-3: /dev/sda vendor: Western Digital model: WD10EADS-42P6B0 
  size: 931.51 GiB 
  ID-4: /dev/sdb vendor: Apple model: HDD WD10EZES-40UFAA0 size: 931.51 GiB 
Partition:
  ID-1: / size: 915.60 GiB used: 85.24 GiB (9.3%) fs: ext4 
  dev: /dev/nvme1n1p2 
Sensors:
  System Temperatures: cpu: 30.9 C mobo: N/A 
  Fan Speeds (RPM): cpu: 0 
  GPU: device: nouveau temp: 24 C fan: 0 device: nouveau temp: 23 C fan: 0 
  device: amdgpu temp: 41 C fan: 1916 
Info:
  Processes: 499 Uptime: 5m Memory: 62.83 GiB used: 4.42 GiB (7.0%) 
  Init: systemd Compilers: gcc: 8.3.0 clang: 8.0.0 Shell: bash v: 5.0.3 
  inxi: 3.0.33 

i have 3 GPU’s build in. In this Order:
1x ASUS STRIX 1080ti
1x AMD Radeon Pro WX3011
1x ASUS STRIX 1080ti.

I posted my problem already on the Manjaro Forum. But there cant nobody help me.

I want to use one of the ASUS Cards for a VM. Iommu works and the Groups are correct. The only problem is to passthrough because of the identical ID of the two nvidia card.

Like in the Manjaro Forum descriped, the Script from the Arch wiki dont work.

Hope somebody can help me finaly.

And sry for my very bad english. Im am nativ German.

This dont work too :frowning:

This is one Error i get:

Somehow i got it working.

i still get the last posted Error in the Screenshot. But it works anyway.

But the start of the VM is very slow. Even the Windows Start Circle loops very laggy.

This Thread can be closed. KVM performance is still slow (only Windows 10 as guest). But this is another problem.

I reinstalled already and switched to Archlinux.

The Solution for my specific passthrough Problem: Simply do nothing. Dont activate iommu with Kernel parameters. Dont do anything. Only(!!) blacklist all Drivers for the nvidia cards. After a restart no driver use the GPU’s and i can add them without problem into virt-manager…