Sorry for my english May be anybody can help me ? I have a 1 monitor witn 2 in (D-SUB+DVI). Host system:
CPU: Intel® Core™ i7-7700K CPU @ 4.20GHz (Kabylake)
MoBo: ASUS PRIME Z270-P
Video0 - Integrated Intel HD Graphics 630
Video1 - Asus GTX960 (UEFI VGA BIOS)
Video2 - Gigbyte GT640 (Non UEFI VGA BIOS)
I dont use integrated graphics. Main host system connected to physiacal monitor over DVI (Video1) and i try passthrough guest with GT640 (Video2) over D-SUB.
OS: Ubuntu 17.10
#> uname -a
Linux land33 4.15.0-041500rc7-generic #201801072330 SMP Sun Jan 7 23:31:29 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux
-
Check IOMMU:
#> dmesg | grep -e DMAR -e IOMMU
[ 0.000000] ACPI: DMAR 0x000000008407CF50 0000A8 (v01 INTEL KBL 00000001 INTL 00000001)
[ 0.000000] DMAR: IOMMU enabled
[ 0.004000] DMAR: Host address width 39
[ 0.004000] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[ 0.004000] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[ 0.004000] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[ 0.004000] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[ 0.004000] DMAR: RMRR base: 0x00000091884000 end: 0x000000918a3fff
[ 0.004000] DMAR: RMRR base: 0x00000093800000 end: 0x00000097ffffff
[ 0.004000] DMAR-IR: IOAPIC id 2 under DRHD base 0xfed91000 IOMMU 1
[ 0.004000] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[ 0.004000] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.004000] DMAR-IR: Enabled IRQ remapping in x2apic mode
[ 0.629101] DMAR: No ATSR found
[ 0.629126] DMAR: dmar0: Using Queued invalidation
[ 0.629209] DMAR: dmar1: Using Queued invalidation
[ 0.629415] DMAR: Setting RMRR:
[ 0.629470] DMAR: Setting identity map for device 0000:00:02.0 [0x93800000 - 0x97ffffff]
[ 0.629498] DMAR: Setting identity map for device 0000:00:14.0 [0x91884000 - 0x918a3fff]
[ 0.629502] DMAR: Prepare 0-16MiB unity mapping for LPC
[ 0.629524] DMAR: Setting identity map for device 0000:00:1f.0 [0x0 - 0xffffff]
[ 0.629664] DMAR: Intel® Virtualization Technology for Directed I/O
-
GRUB default cmd line (GT640 video id and audio id + my pci-e wi-fi ethernet card):
GRUB_CMDLINE_LINUX_DEFAULT=“quiet splash iommu=1 intel_iommu=on iommu=pt vfio-pci.ids=10de:0fc1,10de:0e1b,168c:002e video=efifb:off”
-
file: /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
kvm
kvm_intel
-
file /etc/modprobe.d/kvm.conf
options kvm ignore_msrs=1
-
file /etc/modprobe.d/blacklist.conf (not all, default file + add 1 line)
blacklist nouveau
-
file /etc/modprobe.d/nvidia.conf
softdep nvidia pre: vfio vfio_pci
-
file /etc/modprobe.d/vfio.conf
options vfio-pci ids=10de:0fc1,10de:0e1b
-
file /etc/initramfs-tools
softdep nvidia pre: vfio vfio_pci
vfio
vfio_iommu_type1
vfio_virqfd
options vfio_pci ids=10de:0fc1,10de:0e1b
vfio_pci ids=10de:0fc1,10de:0e1b
vfio_pci
nvidia
kvm
kvm_intel
-
Script (now i run this manually) vfio-pci-override.sh
DEVS="0000:03:00.0 0000:03:00.1"
for DEV in $DEVS; do
echo “vfio-pci” > /sys/bus/pci/devices/$DEV/driver_override
done
modprobe -i vfio-pci
I use GUI and vm-manager for create VMs and try GT640 passthrough to Win10 VM. I create VM as SeaBIOS (BIOS), chipset emulated Q35 and attch host PCI-device over gui. Install guest OS. Remove all Video SPICE. Run guest but i dont see nothing on my phisical monitor over D-SUB (black screen) but i see and hear what VM working.
Next i back add Vido SPICE. Run VM an few past secon see on init screen:
Guest has not initialized the display (yet)
When guest loaded i add old Nvidia driver and see this:
Driver for GT640 in Proerties show “Error code 43”
I googling and special setup oldiest drivers, add few options to VM XML config:
<vendor_id state=‘on’ value=‘whatever’/> and hidden state=‘on’ but have same result. Then guest running i check:
#> dmesg | grep -i vgaarb
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-041500rc7-generic root=UUID=cc5bcfe3-7f3a-45fa-90f2-042fca56e34d ro quiet splash iommu=1 intel_iommu=on vfio-pci.ids=10de:0fc1,10de:0e1b,168c:002e video=efifb:off i915.enable_hd_vgaarb=1 vt.handoff=7
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-041500rc7-generic root=UUID=cc5bcfe3-7f3a-45fa-90f2-042fca56e34d ro quiet splash iommu=1 intel_iommu=on vfio-pci.ids=10de:0fc1,10de:0e1b,168c:002e video=efifb:off i915.enable_hd_vgaarb=1 vt.handoff=7
[ 0.085988] pci 0000:01:00.0: vgaarb: setting as boot VGA device
[ 0.085988] pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none
[ 0.085988] pci 0000:03:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[ 0.085988] pci 0000:03:00.0: vgaarb: bridge control possible
[ 0.085988] pci 0000:01:00.0: vgaarb: bridge control possible
[ 0.085988] vgaarb: loaded
[ 1.766735] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[ 7.051265] nvidia 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem
Check:
#> dmesg | grep -i vfio
[ 0.000000] Command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-041500rc7-generic root=UUID=cc5bcfe3-7f3a-45fa-90f2-042fca56e34d ro quiet splash iommu=1 intel_iommu=on vfio-pci.ids=10de:0fc1,10de:0e1b,168c:002e video=efifb:off i915.enable_hd_vgaarb=1 vt.handoff=7
[ 0.000000] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-4.15.0-041500rc7-generic root=UUID=cc5bcfe3-7f3a-45fa-90f2-042fca56e34d ro quiet splash iommu=1 intel_iommu=on vfio-pci.ids=10de:0fc1,10de:0e1b,168c:002e video=efifb:off i915.enable_hd_vgaarb=1 vt.handoff=7
[ 1.762384] VFIO - User Level meta-driver version: 0.3
[ 1.766735] vfio-pci 0000:03:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
[ 1.784148] vfio_pci: add [10de:0fc1[ffff:ffff]] class 0x000000/00000000
[ 1.804155] vfio_pci: add [10de:0e1b[ffff:ffff]] class 0x000000/00000000
[ 1303.689601] vfio-pci 0000:03:00.0: enabling device (0000 -> 0003)
[ 1303.689815] vfio_ecap_init: 0000:03:00.0 hiding ecap 0x19@0x900
All look as fine but dont work corrctly and i don see nothing on monitor over D-SUB. I try with SPECI/VNC+passthrough GT640 but see only Error code 43 on guest OS. Without SPECI/VNC (only passthrough card GT640) i see black screen over D-SUB but on main scrin host system in GUI in vm-manager i see that VM wrking. After host system boot i run nvidia-smi and it show then i have only one card on host - GTX960 and check:
# lspci -nnk
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM206 [GeForce GTX 960] [10de:1401] (rev a1)
Subsystem: ASUSTeK Computer Inc. GM206 [GeForce GTX 960] [1043:8520]
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_384_drm, nvidia_384
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:0fba] (rev a1)
Subsystem: ASUSTeK Computer Inc. Device [1043:8520]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
03:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK107 [GeForce GT 640] [10de:0fc1] (rev a1)
Subsystem: Gigabyte Technology Co., Ltd GK107 [GeForce GT 640] [1458:353e]
Kernel driver in use: vfio-pci
Kernel modules: nvidiafb, nouveau, nvidia_384_drm, nvidia_384
03:00.1 Audio device [0403]: NVIDIA Corporation GK107 HDMI Audio Controller [10de:0e1b] (rev a1)
Subsystem: Gigabyte Technology Co., Ltd GK107 HDMI Audio Controller [1458:353e]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
OUT find /sys/kernel/iommu_groups/ -type l
/sys/kernel/iommu_groups/7/devices/0000:00:1b.4
/sys/kernel/iommu_groups/15/devices/0000:06:00.0
/sys/kernel/iommu_groups/5/devices/0000:00:17.0
/sys/kernel/iommu_groups/13/devices/0000:03:00.1
/sys/kernel/iommu_groups/13/devices/0000:03:00.0
/sys/kernel/iommu_groups/3/devices/0000:00:14.0
/sys/kernel/iommu_groups/11/devices/0000:00:1d.0
/sys/kernel/iommu_groups/1/devices/0000:01:00.1
/sys/kernel/iommu_groups/1/devices/0000:00:01.0
/sys/kernel/iommu_groups/1/devices/0000:01:00.0
/sys/kernel/iommu_groups/8/devices/0000:00:1c.0
/sys/kernel/iommu_groups/6/devices/0000:00:1b.0
/sys/kernel/iommu_groups/14/devices/0000:05:00.0
/sys/kernel/iommu_groups/4/devices/0000:00:16.0
/sys/kernel/iommu_groups/12/devices/0000:00:1f.3
/sys/kernel/iommu_groups/12/devices/0000:00:1f.4
/sys/kernel/iommu_groups/12/devices/0000:00:1f.2
/sys/kernel/iommu_groups/12/devices/0000:00:1f.0
/sys/kernel/iommu_groups/2/devices/0000:00:02.0
/sys/kernel/iommu_groups/10/devices/0000:00:1c.7
/sys/kernel/iommu_groups/0/devices/0000:00:00.0
/sys/kernel/iommu_groups/9/devices/0000:00:1c.3
WHAT I DO WRONG? May be i need reflash my GT640 with UEFI bios and try run VM witn OVMF what?
I read that:
Passthrough seems to work but no output is displayed
Make sure if you are using virt-manager that UEFI firmware is selected for your virtual machine. Also, make sure you have passed the correct device to the VM.
I try use UEFI firmware (not BIOS) but not have result ;(
I need GT960 use on host! Any body can help me? Sorry for long text. Thanks.