PCIe Passthrough on Proxmox 6.2

Software & Operating Systems VFIO
,
1

Hello Everybody

I’m currently testing Proxmox VE 6.1on my HP ProLiant ML350p Gen8. I’m versy happy with it and am now trying to figure out PCI passthrough. I followed the proxmox guide to pci passthtrough step by step, but i still get the message “No IOMMU detected, please activate it.” when I try to add a PCI device to a VM. VM’s i tried adding to are Ubuntu 20.04 and Fedora 32.

Steps I’ve tried:
1. Checked if VT-d is enabled in BIOS of ML350p: is enabled, I even enabled SR-IOV.
2. updated grub several times with followed reboot after performing the steps in the guide form proxmox. I also ran update-initramfs -u -k all and rebooted.
3. I ran dmesg | grep -e DMAR -e IOMMU and I get output. The only thing different from the guide is that I have xapic instead of x2apic for DMAR-IR.
4. I ran the following script and it returns nothing. Ths probably means my iommu is not enabled.

#!/bin/bash
    shopt -s nullglob
    for d in /sys/kernel/iommu_groups/*/devices/*; do 
        n=${d#*/iommu_groups/*}; n=${n%%/*}
        printf 'IOMMU Group %s ' "$n"
        lspci -nns "${d##*/}"
    done;

HW in the ML350p is 2x Xeon E5-2650 (Sandy Bridge), 2x 750w PSUs, stock NIC. Firmware are all the newest I could find online for the ML350p Gen8. I updated them a while back and have not had problems with the firmware. I’ll list the firmwares below:
grafik

I’m posting this here because I know thet there are some very clever people in this forum, who I hope can help me get going with this.

Thanks in advance and stay safe.

2

I had issues as well (different hw) until I realized there was an IOMMU option in UEFI that I was missing because it was buried 3 menus deep.

3

Was that also on a ProLiant Gen8 Server? I think I checked every menu promted on boot and also in iLO and have not seen such an option.

4

It’s a stretch, but did you remember to do something like this:

GRUB_CMDLINE_LINUX_DEFAULT=“quiet intel_iommu=on”

in /etc/default/grub and then run update-grub? I don’t mean to insult you by pointing out the obvious, but I’ve been there.

5

Thank you Sir. No need to apologize, but yes I’ve done that, this line is in my grub. Only thing I forgot was that linux is case sensitive. I wrote intel_IOMMU=on instead of "intel_iommu=on".
I can now select devices in the “Add: PCI Device” screen of Proxmox.

But now I have this weird message on boot of Proxmox:

grafik
grafik756×288 10.7 KB

That and over iLO it wont take any keyboard inputs anymore. Might be after I also put iommu=pt though. Has anybody got a clue how to fix this?

6

Since it boots and you can now assign PCI devices to VMs, I’m going to guess the ACPI and Bug messages aren’t a problem. I’m also going to guess the keyboard doesn’t work because of the USB controller error, but I don’t have a clue as to why it’s happening.

7

No, what I meant by

was that the cpu and MB are totally different, but on mine, it took me a while to realize there was a menu I had completely missed, thinking that activating VT-d/SVM and SR-IOV was enough- I had to explicitly enable IOMMU for it to work.

8

Ah, alright, thanks for the clarification. Sorry for misunderstanding.
But since that wasn’t the fault the problem has taken care of itself.

The keyboard input over iLO integrated web console is always detected during boot and other operating systems, only on proxmox not anymore.
Could it be because now with IOMMU enabled proxmox (host) doens’t listen to PCI devices anymore?

9

I now updated the iLO4 to FW version 2.73 and with that comes a new HTML5 console. With said console it doesn’t work either.

The only thing that works is when in the PVE webinterface I go on "Shell"on the node. Then I can access the console again.

I also noticed there is an iLO USB Controller that shares an IOMMU group with the Matrox GPU. Could I maybe somehow assign these devices to the host, so that it detects inputs from them again?

grafik
grafik811×86 5.35 KB

Any help would be greatly appreciated.