I’m finally giving it a go to virtual manager with IOMMU & stuff, but I’m having a few problems. Well, there are more than one, but I’d like to sort this one out, first. I’m using Archlinux on an 3700x, B450 tomahawk max, 5700xt (guest) and 750Ti (host) (and 32GB RAM).
Using the boot options vfio-pci.ids=... , ... results in the system booting but showing only the initial “loading kernel” and then it displays nothing else (as if paused). I can hear the system loading (there is one mechanical HD), but I can’t do a thing.
The full boot line is iommu=pt amd_iommu=on vfio... video=vesafb:off,efifb:off . With or without video=, it’s the same. The addresses are for the video and audio from Navi, as listed by lspci and the script to list the devices, found on Archlinux’s page on “PCI_passthrough_via_OVMF” (I can’t include links). The cards are placed: 5700xt in the 1st slot on the MB (closer to the CPU), the 750Ti on the 2nd (the only other one good for video). Is there any disadvantage in swapping them (from the guest’s POV)?
How can I dedicate the 5700xt to the guest? Would this have anything to do with the IOMMU grouping? Because not everything is separated, but Navi is in its own group, alone (same as Navi audio).
Hi, thanks for the reply. That is one of the pages I used for inspiration. In fact, it had higher priority than the Archlinux page precisely due to the similarity of the setup (and a few others). Now, I haven’t used the ACS patch – I’m hoping I can get away without it since Arch can update the kernel anytime – but I’m getting stuck at the “Grub Configuration” part (in that link). I added that line to Grub, except the isolcpus, nohz_full, and rcu_nocbs parts, since they should only have an influence after the booting, when installing the virtual OS.
To be more clear, I only have one monitor and, while booting, it is connected to the 5700xt. I can see the BIOS/UEFI, Grub, and right after booting, when it says loading kernel, then the resolution changes and it says running hooks (or similar) and the 2nd line loading Archlinux .... That’s it. From then, on, the screen freezes, the system seems to load, but if I try to blindly log in and startx, nothing happens. Not even blindly doing Alt+F4 and shutdown. If I switch the monitor to the 750Ti, I get no signal, not even for the BIOS/UEFI.
If I swap the cards in their slots then the IOMMU groups suffer, because the 5700xt will be mingled with the network card, and some others. As it is now, 750Ti is together with those, which is fine since that is meant to be for the host, while 5700xt has its own group.
I suppose my problem is how to select the 750Ti as the default card since the computer is powered on, while keeping the slot arangement.
That’s what I have now, two cables. But it’s an old monitor so I’m using HDMI to SCART and VGA.
I’d really like to know that, too. I looked in the BIOS and I can’t see anywhere any setting like that. There are some about PCI lanes, there’s a graphical display showing the slots and what’s connected, but that’s it. If you know how I can set that up, please let me know. But I wonder why, in that link, there’s nothing about this ever mentioned. Maybe it’s because of the ACS? I wanted to avoid it, but I’ll try it now.
Well, it’s been an adventure, but a bitter one. In short, there was no output at all, this time, so the 5700xt must have been completely isolated, but the 750Ti wasn’t even seen. I swapped the cards and then I could boot, but no startx until I undid the /etc/modprobe.d/vfio.conf and the /etc/defaults/grub (with pacman -Rsc linux-vfio). The script to show the IOMMU groups has the exact same output, and the price was that it took quite a bit to compile everything, I ran out of space on /home resulting in .bash_history being reduced to 4 kB, and a bit of frustration.
My conclusion so far is that I don’t need the ACS patch (since the groups didn’t change, one bit), but I do need to know how can I make the 750Ti the default one but in the 2nd slot, otherwise the 5700xt gets mingled with the network & SATA & co (which I don’t want to pass to the guest OS). The thought that the 750Ti might not be suitable for this did cross my mind, but it is listed as good for the job. Plus, in the link above, that person’s setup is with a 710. But I can’t be sure, so if any of you know more on the subject, please don’t be shy… If you need outputs from lspci or the likes, please let me know.
Hi wertigon, thank you for all the help. I feel just as blind, particularly when staring at a blank screen I remember having visited that page, too, among others, but I’m afraid my current setup is too stubborn. Swapping the cards also means only 4x for the 5700xt, even if it works, so I’d really like not to do that. I would settle for seeing the POST on the 5700xt, then switching to the 750Ti for the OS, even if it will be a bit cumbersome. But, currently, I need to hammer some more. Thank you, again, for the patience. If I manage to get some progress, I’ll either revive this or make a new thread.
I made it! For some reason, the output remained blank, so I added video=dvi-i:e to the kernel boot line, and nouveau to /etc/mkinitcpio.conf, and now it works!
Well, it works to startx and get through the virt-manager setup, but now there’s the other problem: the 5700xt overheats by itself, it gets so hot that it resets the OS, no warnings. This is since day 1. I have no control over it unless I startx and use corectrl. Only in X it uses the low power mode. In POST or no X, no fan starts. And now, because the driver is vfio, corectrl fails to see it. It looks like it can’t keep it in low power mode. Is there any way to control the fans with the vfio driver? (should I open up a new thread?)
