PCI Passthrough Error | QEMU/KVM

Software & Operating Systems Linux
, ,
#1

Hey all! I am attempting to create a Windows 10 Virtual Machine using virt-manager, but can’t pass my wifi card to my machine. This is an issue, as I need a Windows system and a mobile device on the same wireless network to test some software. When attempting to add my PCI hardware through the GUI menu, I am met with an error stating that the host doesn’t support passthrough of host devices. How can I fix this?

My lspci -k output:

00:00.0 Host bridge: Intel Corporation 8th Gen Core 4-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S] (rev 07)
        DeviceName: Onboard - Other
        Subsystem: ASRock Incorporation Device 3e1f
        Kernel driver in use: skl_uncore
        Kernel modules: ie31200_edac
00:01.0 PCI bridge: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) (rev 07)
        Subsystem: ASRock Incorporation Device 1901
        Kernel driver in use: pcieport
00:12.0 Signal processing controller: Intel Corporation Cannon Lake PCH Thermal Controller (rev 10)
        DeviceName: Onboard - Other
        Subsystem: ASRock Incorporation Device a379
        Kernel driver in use: intel_pch_thermal
        Kernel modules: intel_pch_thermal
00:14.0 USB controller: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller (rev 10)
        DeviceName: Onboard - Other
        Subsystem: ASRock Incorporation Device a36d
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci
00:14.2 RAM memory: Intel Corporation Cannon Lake PCH Shared SRAM (rev 10)
        DeviceName: Onboard - Other
        Subsystem: ASRock Incorporation Device a36f
00:16.0 Communication controller: Intel Corporation Cannon Lake PCH HECI Controller (rev 10)
        DeviceName: Onboard - Other
        Subsystem: ASRock Incorporation Device a360
        Kernel driver in use: mei_me
        Kernel modules: mei_me
00:17.0 SATA controller: Intel Corporation Cannon Lake PCH SATA AHCI Controller (rev 10)
        DeviceName: Onboard - SATA
        Subsystem: ASRock Incorporation Device a352
        Kernel driver in use: ahci
00:1d.0 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #11 (rev f0)
        Subsystem: ASRock Incorporation Device a332
        Kernel driver in use: pcieport
00:1d.3 PCI bridge: Intel Corporation Cannon Lake PCH PCI Express Root Port #12 (rev f0)
        Subsystem: ASRock Incorporation Device a333
        Kernel driver in use: pcieport
00:1f.0 ISA bridge: Intel Corporation H370 Chipset LPC/eSPI Controller (rev 10)
        DeviceName: Onboard - Other
        Subsystem: ASRock Incorporation Device a304
00:1f.3 Audio device: Intel Corporation Cannon Lake PCH cAVS (rev 10)
        DeviceName: Onboard - Sound
        Subsystem: ASRock Incorporation Device 288a
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel, snd_soc_skl, snd_sof_pci_intel_cnl
00:1f.4 SMBus: Intel Corporation Cannon Lake PCH SMBus Controller (rev 10)
        DeviceName: Onboard - Other
        Subsystem: ASRock Incorporation Device a323
        Kernel driver in use: i801_smbus
        Kernel modules: i2c_i801
00:1f.5 Serial bus controller: Intel Corporation Cannon Lake PCH SPI Controller (rev 10)
        DeviceName: Onboard - Other
        Subsystem: ASRock Incorporation Device a324
        Kernel driver in use: intel-spi
        Kernel modules: intel_spi_pci
01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Lexa PRO [Radeon 540/540X/550/550X / RX 540X/550/550X] (rev c7)
        Subsystem: ASRock Incorporation Device 5001
        Kernel driver in use: amdgpu
        Kernel modules: amdgpu
01:00.1 Audio device: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X]
        Subsystem: ASRock Incorporation Device aae0
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
        Subsystem: ASRock Incorporation Motherboard (one of many)
        Kernel driver in use: r8169
        Kernel modules: r8169
03:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8821CE 802.11ac PCIe Wireless Network Adapter
        Subsystem: AzureWave Device 3043
        Kernel driver in use: rtw_8821ce
        Kernel modules: rtw88_8821ce

My lspci -nn output:

00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core 4-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S] [8086:3e1f] (rev 07)
00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake PCH SATA AHCI Controller [8086:a352] (rev 10)
00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #11 [8086:a332] (rev f0)
00:1d.3 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #12 [8086:a333] (rev f0)
00:1f.0 ISA bridge [0601]: Intel Corporation H370 Chipset LPC/eSPI Controller [8086:a304] (rev 10)
00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
01:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Lexa PRO [Radeon 540/540X/550/550X / RX 540X/550/550X] [1002:699f] (rev c7)
01:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Baffin HDMI/DP Audio [Radeon RX 550 640SP / RX 560/560X] [1002:aae0]
02:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
03:00.0 Network controller [0280]: Realtek Semiconductor Co., Ltd. RTL8821CE 802.11ac PCIe Wireless Network Adapter [10ec:c821]

My grub.cfg file:

#
# DO NOT EDIT THIS FILE
#
# It is automatically generated by grub-mkconfig using templates
# from /etc/grub.d and settings from /etc/default/grub
#

### BEGIN /etc/grub.d/00_header ###
insmod part_gpt
insmod part_msdos
if [ -s $prefix/grubenv ]; then
  load_env
fi
if [ "${next_entry}" ] ; then
   set default="${next_entry}"
   set next_entry=
   save_env next_entry
   set boot_once=true
else
   set default="0"
fi

if [ x"${feature_menuentry_id}" = xy ]; then
  menuentry_id_option="--id"
else
  menuentry_id_option=""
fi

export menuentry_id_option

if [ "${prev_saved_entry}" ]; then
  set saved_entry="${prev_saved_entry}"
  save_env saved_entry
  set prev_saved_entry=
  save_env prev_saved_entry
  set boot_once=true
fi

function savedefault {
  if [ -z "${boot_once}" ]; then
    saved_entry="${chosen}"
    save_env saved_entry
  fi
}

function load_video {
  if [ x$feature_all_video_module = xy ]; then
    insmod all_video
  else
    insmod efi_gop
    insmod efi_uga
    insmod ieee1275_fb
    insmod vbe
    insmod vga
    insmod video_bochs
    insmod video_cirrus
  fi
}

if [ x$feature_default_font_path = xy ] ; then
   font=unicode
else
insmod part_msdos
insmod ext2
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  410846cc-9432-4b2b-92fe-55c5c565602b
else
  search --no-floppy --fs-uuid --set=root 410846cc-9432-4b2b-92fe-55c5c565602b
fi
    font="/usr/share/grub/unicode.pf2"
fi

if loadfont $font ; then
  set gfxmode=auto
  load_video
  insmod gfxterm
  set locale_dir=$prefix/locale
  set lang=en_US
  insmod gettext
fi
terminal_input console
terminal_output gfxterm
insmod part_msdos
insmod ext2
set root='hd0,msdos1'
if [ x$feature_platform_search_hint = xy ]; then
  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  410846cc-9432-4b2b-92fe-55c5c565602b
else
  search --no-floppy --fs-uuid --set=root 410846cc-9432-4b2b-92fe-55c5c565602b
fi
insmod gfxmenu
loadfont ($root)/boot/grub/themes/EndeavourOS/unifont-regular-16.pf2
insmod png
set theme=($root)/boot/grub/themes/EndeavourOS/theme.txt
export theme
if [ x$feature_timeout_style = xy ] ; then
  set timeout_style=menu
  set timeout=5
# Fallback normal timeout code in case the timeout_style feature is
# unavailable.
else
  set timeout=5
fi
### END /etc/grub.d/00_header ###

### BEGIN /etc/grub.d/10_linux ###
menuentry 'EndeavourOS, on linux-zen' --class endeavouros --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-zen-advanced-410846cc-9432-4b2b-92fe-55c5c565602b' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_msdos
	insmod ext2
	set root='hd0,msdos1'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  410846cc-9432-4b2b-92fe-55c5c565602b
	else
	  search --no-floppy --fs-uuid --set=root 410846cc-9432-4b2b-92fe-55c5c565602b
	fi
	echo	'Loading kernel linux-zen ...'
	linux	/boot/vmlinuz-linux-zen root=UUID=410846cc-9432-4b2b-92fe-55c5c565602b rw  resume=UUID=c96fd141-6fba-4e51-901b-5e3e734d6486 loglevel=3 nowatchdog nvme_load=YES
	echo	'Loading initial ramdisk ...'
	initrd	/boot/intel-ucode.img /boot/initramfs-linux-zen.img
}
menuentry 'EndeavourOS, on linux-zen (fallback initramfs)' --class endeavouros --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-zen-fallback-410846cc-9432-4b2b-92fe-55c5c565602b' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_msdos
	insmod ext2
	set root='hd0,msdos1'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  410846cc-9432-4b2b-92fe-55c5c565602b
	else
	  search --no-floppy --fs-uuid --set=root 410846cc-9432-4b2b-92fe-55c5c565602b
	fi
	echo	'Loading kernel linux-zen ...'
	linux	/boot/vmlinuz-linux-zen root=UUID=410846cc-9432-4b2b-92fe-55c5c565602b rw  resume=UUID=c96fd141-6fba-4e51-901b-5e3e734d6486 loglevel=3 nowatchdog nvme_load=YES
	echo	'Loading initial ramdisk ...'
	initrd	/boot/intel-ucode.img /boot/initramfs-linux-zen-fallback.img
}
menuentry 'EndeavourOS, on linux' --class endeavouros --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-advanced-410846cc-9432-4b2b-92fe-55c5c565602b' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_msdos
	insmod ext2
	set root='hd0,msdos1'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  410846cc-9432-4b2b-92fe-55c5c565602b
	else
	  search --no-floppy --fs-uuid --set=root 410846cc-9432-4b2b-92fe-55c5c565602b
	fi
	echo	'Loading kernel linux ...'
	linux	/boot/vmlinuz-linux root=UUID=410846cc-9432-4b2b-92fe-55c5c565602b rw  resume=UUID=c96fd141-6fba-4e51-901b-5e3e734d6486 loglevel=3 nowatchdog nvme_load=YES
	echo	'Loading initial ramdisk ...'
	initrd	/boot/intel-ucode.img /boot/initramfs-linux.img
}
menuentry 'EndeavourOS, on linux (fallback initramfs)' --class endeavouros --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-linux-fallback-410846cc-9432-4b2b-92fe-55c5c565602b' {
	load_video
	set gfxpayload=keep
	insmod gzio
	insmod part_msdos
	insmod ext2
	set root='hd0,msdos1'
	if [ x$feature_platform_search_hint = xy ]; then
	  search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1  410846cc-9432-4b2b-92fe-55c5c565602b
	else
	  search --no-floppy --fs-uuid --set=root 410846cc-9432-4b2b-92fe-55c5c565602b
	fi
	echo	'Loading kernel linux ...'
	linux	/boot/vmlinuz-linux root=UUID=410846cc-9432-4b2b-92fe-55c5c565602b rw  resume=UUID=c96fd141-6fba-4e51-901b-5e3e734d6486 loglevel=3 nowatchdog nvme_load=YES
	echo	'Loading initial ramdisk ...'
	initrd	/boot/intel-ucode.img /boot/initramfs-linux-fallback.img
}

### END /etc/grub.d/10_linux ###

### BEGIN /etc/grub.d/20_linux_xen ###
### END /etc/grub.d/20_linux_xen ###

### BEGIN /etc/grub.d/30_os-prober ###
### END /etc/grub.d/30_os-prober ###

### BEGIN /etc/grub.d/30_uefi-firmware ###
### END /etc/grub.d/30_uefi-firmware ###

### BEGIN /etc/grub.d/40_custom ###
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.
### END /etc/grub.d/40_custom ###

### BEGIN /etc/grub.d/41_custom ###
if [ -f  ${config_directory}/custom.cfg ]; then
  source ${config_directory}/custom.cfg
elif [ -z "${config_directory}" -a -f  $prefix/custom.cfg ]; then
  source $prefix/custom.cfg
fi
### END /etc/grub.d/41_custom ###

Any help would be greatly appreciated. Thanks in advance!

#2

This usually means the IOMMU isn’t active.

It needs to be turned on both in the bios (vt-d for intel, IOMMU in AMD, usually) and as a kernel argument for most systems. (though I think recently AMD has it enabled by default)

Also, you’re using the zen kernel. I’m not familiar with using that with kvm. Out of my wheelhouse.

#3

Thanks for the fast reply. While I did confirm that VT-d was enabled in the BIOS, I’m not sure that it is enabled as a kernel argument, nor do I know how to confirm this. Are certain kernels superior to others in regards to virtualization?

#4

sudo dmesg | grep iommu

Should something in there.

All I know about the Zen kernel is that it’s foreign to me and I stay away from it for that reason. That’s not an endorsement, nor the opposite, just me admitting my lack of knowledge. I may be able to help otherwise though.

#5

Gotcha. Here’s the output:

[    0.213496] iommu: Default domain type: Translated 
[    0.213497] iommu: DMA domain TLB invalidation policy: lazy mode
#6

okay, try this:

sudo dmesg | grep DMAR

Usually it shows a list of groups it’s created.

#7

Here you are:

[    0.005851] ACPI: DMAR 0x000000008E8B9B78 000070 (v01 INTEL  EDK2     00000002      01000013)
[    0.005868] ACPI: Reserving DMAR table memory at [mem 0x8e8b9b78-0x8e8b9be7]
[    0.074721] DMAR: Host address width 39
[    0.074722] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.074725] DMAR: dmar0: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.074727] DMAR: RMRR base: 0x0000008f00a000 end: 0x0000008f253fff
[    0.074729] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 0
[    0.074730] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.074730] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.077889] DMAR-IR: Enabled IRQ remapping in x2apic mode
#8

Hmmm, looks like your IOMMU isn’t enabled.

cat /proc/cmdline?

Are you on Intel or AMD?

#9

I’m using Intel. lscpu output:

Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         39 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  4
  On-line CPU(s) list:   0-3
Vendor ID:               GenuineIntel
  Model name:            Intel(R) Core(TM) i3-9100F CPU @ 3.60GHz
    CPU family:          6
    Model:               158
    Thread(s) per core:  1
    Core(s) per socket:  4
    Socket(s):           1
    Stepping:            10
    CPU(s) scaling MHz:  19%
    CPU max MHz:         4200.0000
    CPU min MHz:         800.0000
    BogoMIPS:            7200.00
    Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology
                          nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx est tm2 ssse3 sdbg fma cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm abm 3dnowpr
                         efetch cpuid_fault epb invpcid_single pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid mpx rdseed adx smap clflushopt intel_pt xsaveopt x
                         savec xgetbv1 xsaves dtherm ida arat pln pts hwp hwp_notify hwp_act_window hwp_epp md_clear flush_l1d arch_capabilities
Virtualization features: 
  Virtualization:        VT-x
Caches (sum of all):     
  L1d:                   128 KiB (4 instances)
  L1i:                   128 KiB (4 instances)
  L2:                    1 MiB (4 instances)
  L3:                    6 MiB (1 instance)
NUMA:                    
  NUMA node(s):          1
  NUMA node0 CPU(s):     0-3
Vulnerabilities:         
  Itlb multihit:         KVM: Mitigation: VMX disabled
  L1tf:                  Mitigation; PTE Inversion; VMX conditional cache flushes, SMT disabled
  Mds:                   Mitigation; Clear CPU buffers; SMT disabled
  Meltdown:              Mitigation; PTI
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Retpolines, IBPB conditional, IBRS_FW, STIBP disabled, RSB filling
  Srbds:                 Mitigation; Microcode
  Tsx async abort:       Not affected

Cat output:

BOOT_IMAGE=/boot/vmlinuz-linux-zen root=UUID=410846cc-9432-4b2b-92fe-55c5c565602b rw resume=UUID=c96fd141-6fba-4e51-901b-5e3e734d6486 loglevel=3 nowatchdog nvme_load=YES
#10

Ok, your CPU has IOMMU support, but it’s not enabled for some reason.

There are multiple “levels” of Intel virtualization.

You’ll notice in your “Virtualization Features” section of the lscpu output, it say VT-x. You’re looking for VT-d.

I would peruse your bios and add intel_iommu=pt to the grub flags.

#11

I see, where do I append that in the grub configuration file(s)?

#12

/etc/default/grub

GRUB_CMDLINE_LINUX_DEFAULT variable.

#13

Odd, that variable doesn’t exist. My grub.cfg file and my /default/grub file are the same.

#14

What docs have you been following to try PCI passthrough out?
Any decent docs should list the steps to enable IOMMU on each platform, and usually has a script to detcst whether it’s working.

1 Like
#15

that’s incredibly strange.

I have to step out for probably the rest of the day, for work. I can return to this later.

#16

I skimmed through this page to append GRUB_CMDLINE_LINUX_DEFAULT to my grub file, which I seem to have never done, because as I stated, it’s the same as grub.cfg.

#17

Take your time, thank you for your help so far.

1 Like
#18

Could this be the auto vs full enable thing? On some boards you have to switch IOMMU from “auto” to “on” otherwise it won’t work properly.

#19

I don’t believe that is the issue in this case. My motherboard (ASRock) doesn’t have an IOMMU option to my knowledge, instead simply containing a VT-d toggle, which I have enabled.

#20

That is correct for Intel.

I kinda wrote the book on this topic, this should help: