Enabling IOMMU on AMD build

Software & Operating Systems Linux
1

I have an AMD Ryzen 7 5800X CPU and a AMD ATI Radeon RX 6700 XT GPU, and I’d like to take advantage of them in a Windows virtual machine for certain videogames and software. My motherboard is an ASRock B550 Phantom Gaming 4/ac, but I can’t find any way to enable IOMMU on it.

Running dmesg | grep -e IOMMU outputs:

[    2.956817] AMD-Vi: AMD IOMMUv2 functionality not available on this system - This is not a bug.

but I’ve been told my setup should support it. For what it’s worth, I’m running Gentoo with the sys-kernel/gentoo-kernel-bin kernel. Any advice is much appreciated.

2

Is that a B550 chipset or the 450 just updated a little bit

Start with that

3

How can I tell?

4

I forget if the newer denomination was b550m or something else but theres an askgn video on it. Google b550 vs b450

5

Yeah, it seems to be a B550 chipset motherboard (according to Amazon and Newegg), not B550m.

6

That;ll probably be why.

7

Is it impossible with my motherboard, in that case? :frowning:

8

Just to be sure, in the BIOS you checked if the IOMMU toggle is present at this location?

Advanced → AMD CBS → NBIO Common Options

Default options are probably “Auto” which means disabled in 99 % of the cases.

1 Like
9

There it was! It was disabled, thank you so much for the help, I couldn’t find it anywhere!

2 Likes
10

No problem, have been troubleshooting so many AMD platforms for the past few years that I almost know my way around the various UEFIs blindly :face_vomiting:

2 Likes
11

Okay, so now after adding the follow kernel parameters to my rEFInd config:

"... iommu=pt pcie_acs_override=downstream,multifunction ..."

and the following to my /etc/modprobe.d/vfio.conf:

options vfio-pci ids=1002:73df,1002:ab28

I try booting up and get

* Loading kernel modules ...
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio/vfio.ko
modprobe: ERROR: could not insert 'vfio': Module already in kernel
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio/vfio_iommu_type1.ko
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio_virqfd.ko
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio/pci/vfio-pci-core.ko
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio/pci/vfio-pci.ko ids=1002:73df,1002:ab28

before it hangs and doesn’t finish booting. :confused: No idea what to do here. The IDs on my vfio.conf file come from here (lspci -nn):

0b:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 22 [Radeon RX 6700/6700 XT / 6800M] [1002:73df] (rev c1)
0b:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 21 HDMI Audio [Radeon RX 6800/6800 XT / 6900 XT] [1002:ab28]

After temporarily removing the vfio kernel parameters from my rEFInd config and /etc/modprobe.d/vfio.conf so I can boot into my desktop again, I was at least able to prove that IOMMU was enabled (dmesg | grep -e IOMMU):

[    0.184879] pci 0000:00:00.2: AMD-Vi: IOMMU performance counters supported
[    0.187119] pci 0000:00:00.2: AMD-Vi: Found IOMMU cap 0x40
[    0.187725] perf/amd_iommu: Detected AMD IOMMU #0 (2 banks, 4 counters/bank).
[    2.978401] AMD-Vi: AMD IOMMUv2 loaded and initialized

Any avice is greatly appreciated, I feel so close to finally finishing up here. :sob:

12

While I’m not competent enough to help further I’d suggest you create a dedicated thread to specifically lure in the crowd that knows about Linux VM hosts.

1 Like
14

Normally you not only have the file in /etc/modprobe.d/vfio.conf, but also load the modules in /etc/mkinitcpio.conf or via a *.conf file in /etc/modules-load.d/. I mean the place where you load the modules vfio_pci vfio vfio_iommu_type1 vfio_virqfd. Can you please post this file?

1 Like
15

I don’t have a /etc/mkinitcpio.conf (I remember looking it up and coming away with the idea that it was Arch-specific), but neither do I have even a /etc/modules-load.d/ directory! I guess I should make it, but I don’t know what to put inside…

16

I figured it out, but, after re-adding my VFIO configuration in /boot/refind_linux.conf and /etc/modprobe.d/vfio.conf before making /etc/modules-load.d/vfio.conf:

# load vfio modules
vfio-pci vfio vfio_iommu_type1 vfio_virqfd

and rebooting… my computer froze again, this time on:

* Loading module vfio-pci ...

17
  • remove the pcie_acs_override flag in the boot params
  • have you blacklisted amdgpu ?
18

After doing both of these, I get this error again. :sob:

* Loading kernel modules ...
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio/vfio.ko
modprobe: ERROR: could not insert 'vfio': Module already in kernel
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio/vfio_iommu_type1.ko
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio_virqfd.ko
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio/pci/vfio-pci-core.ko
insmod /lib/modules/5.15.12-gentoo-dist/kernel/drivers/vfio/pci/vfio-pci.ko ids=1002:73df,1002:ab28

It just freezes here on boot…

19

can you post your vfio kernel compile settings?

root@pve:~# cat /boot/config-`uname -r` | grep -i vfio
CONFIG_KVM_VFIO=y
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO_VIRQFD=m
CONFIG_VFIO=m
CONFIG_VFIO_NOIOMMU=y
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_VGA=y
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_VFIO_PCI_IGD=y
CONFIG_VFIO_MDEV=m
CONFIG_VFIO_MDEV_DEVICE=m
# CONFIG_SAMPLE_VFIO_MDEV_MTTY is not set
# CONFIG_SAMPLE_VFIO_MDEV_MDPY is not set
# CONFIG_SAMPLE_VFIO_MDEV_MDPY_FB is not set
# CONFIG_SAMPLE_VFIO_MDEV_MBOCHS is not set
20

yessir!

root:~# cat /boot/config-`uname -r` | grep -i vfio
CONFIG_KVM_VFIO=y
CONFIG_VFIO=m
CONFIG_VFIO_IOMMU_TYPE1=m
CONFIG_VFIO_VIRQFD=m
CONFIG_VFIO_NOIOMMU=y
CONFIG_VFIO_PCI_CORE=m
CONFIG_VFIO_PCI_MMAP=y
CONFIG_VFIO_PCI_INTX=y
CONFIG_VFIO_PCI=m
CONFIG_VFIO_PCI_VGA=y
CONFIG_VFIO_PCI_IGD=y
CONFIG_VFIO_MDEV=m
21

Silly question, is the GPU configured in vfio the only one you have in the PC?

1 Like