Seemingly unable to do VFIO with a HD7950

Hey there

I thought I’d try this whole VFIO thing with an older GPU to see if it works. Ideally I’d get a low power GPU to run my linux desktop and use my 1660Ti for VFIO but I need to see this working before I can justify buying yet another GPU.

So, I’m gonna post my specs here

Distro: Manjaro
CPU: i5-6500 OCd
GPU (host): 1660Ti
GPU (vm): HD7950
RAM: 16GB 2666MHz
Mobo: MSI Z170a PC MATE

I have set up all the preliminary stuff, Intel VT-d in bios, kernel parameters etc.

shopt -s nullglob
for g in /sys/kernel/iommu_groups/*; do
    echo "IOMMU Group ${g##*/}:"
    for d in $g/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done;
done;

results in:

IOMMU Group 14:
        04:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti PRO [Radeon HD 7950/8950 OEM / R9 280] [1002:679a]
        04:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti HDMI Audio [Radeon HD 7870 XT / 7950/7970] [1002:aaa0]

/etc/modprobe.d/vfio.conf contains "options vfio-pci ids=1002:679a,1002:aaa0"

/etc/mkinitcpio.conf contains MODULES="vfio vfio_iommu_type1 vfio_pci vfio_virqfd" as well as HOOKS="base udev autodetect modconf block keyboard keymap filesystems"

I regenerated initramfs after adding modifying these files with
mkinitcpio -g /boot/linux-custom.img

/etc/libvirt/qemu.conf includes

nvram = [ “/usr/share/ovmf/x64/OVMF_CODE.fd:/usr/share/ovmf/x64/OVMF_VARS.fd” ]

I also attempted to pass the device IDs as kernel parameters.

dmesg | grep -i vfio results in

> 82120] Kernel command line: BOOT_IMAGE=/boot/vmlinuz-5.3-x86_64 root=UUID=ce71f013-0d5d-4f05-a55e-d7c87f03efed rw intel_idle
> .max_cstate=0 intel_iommu=on iommu=pt vfio-pci.ids=1002:679a,1002:aaa0
> [  155.007526] VFIO - User Level meta-driver version: 0.3
> [  155.016667] vfio_pci: add [1002:679a[ffffffff:ffffffff]] class 0x000000/00000000
> [  155.016670] vfio_pci: add [1002:aaa0[ffffffff:ffffffff]] class 0x000000/00000000
> [  155.574294] vfio-pci 0000:04:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
> [  156.643322] vfio-pci 0000:04:00.0: vfio_ecap_init: hiding ecap [email protected]
> [  156.643329] vfio-pci 0000:04:00.0: vfio_ecap_init: hiding ecap [email protected]
> [  208.826445] vfio-pci 0000:04:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=io+mem:owns=none
> [[email protected] ~]$

however, lspci -nnk
results in

04:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti PRO [Radeon HD 7950/8950 OEM / R9 280] [1002:679a]
Subsystem: Gigabyte Technology Co., Ltd Tahiti PRO [Radeon HD 7950/8950 OEM / R9 280] [1458:254c]
Kernel driver in use: radeon
Kernel modules: radeon, amdgpu
04:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Tahiti HDMI Audio [Radeon HD 7870 XT / 7950/7970] [1002:aaa0]
Subsystem: Gigabyte Technology Co., Ltd Tahiti HDMI Audio [Radeon HD 7870 XT / 7950/7970] [1458:aaa0]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel

It should be saying Kernel driver in use: vfio-pci for both of them.

I installed windows successfully, then removed the virtual display, and added the AMD PCI devices for video and audio.

when attempting to run the VM no display pops up.

Additionally, I followed the steps in the arch wiki article PCI passthrough via OVMF to pass keyboard / mouse via Evdev since I only have one of each. This however has not worked since after I hit run on the VM the mouse and kb stop working entirely. This might be due to there being no graphical window, but I’m not sure.

Any help or suggestions are welcome, I’m trying to set this up for the first time so go easy on me if I missed anything obvious please.

Thank you

EDIT
I added softdep radeon pre: vfio-pci to my /etc/modprobe.d/vfio.conf which resulted in the VGA compatible controller reporting vfio-pci as the Kernel driver in use.
The audio device still says snd_hda_intel and I’m still not getting a video in my VM

1 Like

Hi again.

I’ve discovered that passing the vfio information as kernel parameters is not working fully for me. I removed them and only defined the ids through /etc/modprobe.d/vfio.conf

I managed to get both my video card and it’s audio device to use the vfio-pci driver

However, when starting the VM I still get the error graphical console not configured for guest

In my hardware tab everything looks fine

As a sidenote, I got my keyboard/mouse to work and switch perfectly with the Evdev solution.

The graphical console not configured is normal - you use a display attached to the actual GPU for output.

Under the performance bit, the graph should show peaks and troughs to show activity, so you know it’s not just flat lining.

Oh. I’m dumb lmao. I thought it would work just like as a picture in picture. Now I see why people need all those physical kvm switches.

Thanks!

Not dumb at all.

VMM provides that screen to interract with VM’s, but they are removed in favour of direct output on passed through GPU.

Most VM managers don’t allow for the GPU passthrough, and use that screen for the VM’s

I see.

What I don’t see is any output GPU, the monitor just looks for signals, doesn’t find any and goes black. Tried two different monitors and two different HDMI cables.

I dont know if this is considered flatlining or not

:thinking:

Huh, looks like it tries to go thorugh boot, but stops for some reason.

I’m not sure why you aren’t getting any output though