Are these Problems a deathsentence for my motherboard? Proxmox 2080ti Passthrough on 9700k & Z370P-D3

Software & Operating Systems VFIO
, ,
1

Hello everyone, i have been trying different guides (beginners guide on /r/homelab, craftcomputing, proxmox official) with my 9700k on a Z370P-D3 running a fresh proxmox 7.0.8 to pass through my 2080ti. However, proxmox still shows “No IOMMU detected, please activate it. See Documentation for further information.”

At this point i dont know where to look for new troubleshooting steps - i don’t know what’s not working.

I have enabled VT-D in BIOS, however there is no separate IOMMU setting.

I have tried both common grubline options.

I have tried blacklisting drivers and adding device ids /etc/modprobe.d/vfio.conf disable_vga (first 2 and all 4)

My lspci -t looks pretty nicely separated (GPU is 01:00.0 to 01:00.4 with gpu, sound, hub and USB). Does this already show iommu groups or is this just the bus layout? Is it possible my Board doesnt support iommu at all and i’ve been wasting my time encouraged by this nice layout?

    -[0000:00]-+-00.0
               +-01.0-[01]--+-00.0
               |            +-00.1
               |            +-00.2
               |            \-00.3
               +-02.0
               +-08.0
               +-14.0
               +-16.0
               +-17.0
               +-1b.0-[02]--
               +-1b.2-[03]--
               +-1b.3-[04]--
               +-1b.4-[05]--
               +-1c.0-[06]--
               +-1c.2-[07]----00.0
               +-1c.3-[08]----00.0
               +-1c.4-[09]--
               +-1d.0-[0a]--
               +-1f.0
               +-1f.2
               +-1f.3
               \-1f.4

The commonly suggested troubleshooting dmesg | grep shows the following:

dmesg | grep -e DMAR -e IOMMU
[    0.007009] ACPI: DMAR 0x00000000B8F569B8 0000A8 (v01 ALASKA A M I    00000001 INTL 00000001)
[    0.007037] ACPI: Reserving DMAR table memory at [mem 0xb8f569b8-0xb8f56a5f]
[    0.114700] DMAR: Host address width 39
[    0.114701] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.114705] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.114708] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.114710] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.114712] DMAR: RMRR base: 0x000000b97dd000 end: 0x000000b9a26fff
[    0.114714] DMAR: RMRR base: 0x000000bb000000 end: 0x000000bf7fffff
[    0.114715] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.114717] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.114718] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.116142] DMAR-IR: Enabled IRQ remapping in x2apic mode

I also found this script in another troubleshooting post, yielding the following

for a in /sys/kernel/iommu_groups/*; do find $a -type l; done | sort --version-sort
find: ‘/sys/kernel/iommu_groups/*’: No such file or directory.

Thank you very much for any hint what to look for! If this is a problem with my motherboard, i would be glad for any hints of compatlible motherboards you have gotten to work.

2

I’ve never used proxmox but the basic principles should be the same.
For an Intel system, you’ll need to enable VT-d in the BIOS and set the intel_iommu=on kernel parameter. I also have iommu=pt kernel parameter set.

You can check the parameters used by your system at boot with cat /proc/cmdline (at least on my system this works)

I’m sure if you can enable VT-d on your motherboard it should be able to do IOMMU. Your specs are quite recent so I think you should be fine.

Try this script from the Arch wiki to check IOMMU groups:

#!/bin/bash
shopt -s nullglob
for g in `find /sys/kernel/iommu_groups/* -maxdepth 0 -type d | sort -V`; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;
3

Hi, thanks for the pointers. I have already set up grub, vfio modules, interrupt remapping and added the device ids to vfio.conf.

The output for a very similar script is already in the post, there are no iommu folders.

4

No IOMMU folders is due to IOMMU not being enabled.
I would go through and check the grub cmdline options are correct and regenerate grub again with update-grub as root

Reboot and check dmesg again with dmesg | grep -i -e DMAR -e IOMMU
Look for Intel-IOMMU: enabled in the output.

You could also try updating your BIOS to the latest firmware version, might be worth a shot.

5

Hi, thanks for the suggestion, i did it with a fresh install, but all i get is same old:

root@homelab:~# dmesg | grep -e DMAR -e IOMMU
[    0.006962] ACPI: DMAR 0x00000000B8F569B8 0000A8 (v01 ALASKA A M I    00000001 INTL 00000001)
[    0.006990] ACPI: Reserving DMAR table memory at [mem 0xb8f569b8-0xb8f56a5f]
[    0.114553] DMAR: Host address width 39
[    0.114554] DMAR: DRHD base: 0x000000fed90000 flags: 0x0
[    0.114558] DMAR: dmar0: reg_base_addr fed90000 ver 1:0 cap 1c0000c40660462 ecap 19e2ff0505e
[    0.114561] DMAR: DRHD base: 0x000000fed91000 flags: 0x1
[    0.114563] DMAR: dmar1: reg_base_addr fed91000 ver 1:0 cap d2008c40660462 ecap f050da
[    0.114565] DMAR: RMRR base: 0x000000b97dd000 end: 0x000000b9a26fff
[    0.114567] DMAR: RMRR base: 0x000000bb000000 end: 0x000000bf7fffff
[    0.114569] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 1
[    0.114570] DMAR-IR: HPET id 0 under DRHD base 0xfed91000
[    0.114571] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[    0.115988] DMAR-IR: Enabled IRQ remapping in x2apic mode
6

In case anyone stumbles upon this in the future: The problem has been solved over at proxmox forums.

2 Likes