Managed to disable the first GPU and reserve it for the guest while using the card from my 3rd pci slot (a titan x) for the host.
The problem is that I have no video output from the guest gpu when starting the VM.
dmesg | grep vfio
shows something interesting:
[ 947.207804] vfio_ecap_init: 0000:42:00.0 hiding ecap 0x1e@0x258
[ 947.207826] vfio_ecap_init: 0000:42:00.0 hiding ecap 0x19@0x900
[ 947.210200] vfio-pci 0000:42:00.0: BAR 3: can't reserve [mem 0x90000000-0x91ffffff 64bit pref]
[ 950.413347] vfio-pci 0000:42:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
[ 950.413378] vfio-pci 0000:42:00.0: Invalid PCI ROM header signature: expecting 0xaa55, got 0xffff
lspci
:
42:00.0 VGA compatible controller [0300]: NVIDIA Corporation Device [10de:1e07] (rev a1)
Subsystem: NVIDIA Corporation Device [10de:1e07]
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia
42:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:10f7] (rev a1)
Subsystem: NVIDIA Corporation Device [10de:1e07]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
42:00.2 USB controller [0c03]: NVIDIA Corporation Device [10de:1ad6] (rev a1)
Subsystem: NVIDIA Corporation Device [10de:1e07]
Kernel driver in use: vfio-pci
42:00.3 Serial bus controller [0c80]: NVIDIA Corporation Device [10de:1ad7] (rev a1)
Subsystem: NVIDIA Corporation Device [10de:1e07]
Kernel driver in use: vfio-pci
0a:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM200 [GeForce GTX TITAN X] [10de:17c2] (rev a1)
Subsystem: ASUSTeK Computer Inc. GM200 [GeForce GTX TITAN X] [1043:8539]
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia
0a:00.1 Audio device [0403]: NVIDIA Corporation GM200 High Definition Audio [10de:0fb0] (rev a1)
Subsystem: ASUSTeK Computer Inc. GM200 High Definition Audio [1043:8539]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel