Return to Level1Techs.com

Problems encountered Starting a VM under Mint with KVM&QEMU

vfio
#22

Libvirt is sorted. Managed to make the vm and install windows. I think the gpu is also there, allthough i havent tried to install its drivers yet, Becuase i have encountered yet another problem.

I want to bind one of my usb 3.0 controllers (i have 4 of them on a pci card) to vfio so that i can give them to the vm.

Giving them just like that, makes the vm not see its boot drive.

I learned its a bit tricky to remove the xhci bind and bind it to vfio, so first i tried to manualy unbind from XHCI and the manually rebind to VFIO

According to this you can manualy bind unbind devices from their drivers


so thats what someone pointed here

I proceeded accordingly

echo -n 0000:43:00.0 > /sys/bus/pci/devices/0000:43:00.0/driver/unbind
to unbind
and then
echo -n 0000:43:00.0 > /sys/bus/pci/drivers/vfio-pci/bind

Here is the tricky part.
Unbinding always works.
i can see with
lspci -nnv that the device is no longer bound to xhci

but when i try to bind it to vfio there is a problem.

Sometimes it works.
Sometimes it tells me no such device.

How the faq can that be at all, that sometimes its working, and sometimes it tells me theres no device ?

Is this unbinding not always the same, in sense that it sometimes releases the devices somewhere else, from which the bind command no longer is able to find it.

This doesnt make sense at all.
Also this solution is not permanent, as after the restart, the controller is still bound to xhci.

I could try to add it to iniramfs, through the script that bounded me the gpu to vfio, but because it is inconsistent, and it doesnt alwazs work (the rebinding part) i cant rely on it.

Perhaps any ideeas why sometimes i get no such device ?

0 Likes

#23

Now this was totally unexpected.
Scouring the Internet i found this script
https://raw.githubusercontent.com/andre-richter/vfio-pci-bind/master/vfio-pci-bind.sh

this can be downloaded, made executable, and run with command line Argument the device which you want to bind to vfio
like this

sudo ./vfio-pci-bind.sh 0000:43:00.0

and surprise, this device is bound to vfio.
Holy faq !

Most interesting fact to note, is that this script would also work for binding one of two similar gpus to vfio, or whatever pci device you Need bounded to VFIO.

SHIT, this script is FK GOLDEN.

0 Likes

#24

Finally manage to get USB controllers and gpu to bind to VFIO. I used a combination of stuff, first unbinding the usb controllers, then using driver override

echo -n 0000:43:00.0 > /sys/bus/pci/devices/0000:43:00.0/driver/unbind
echo -n 0000:44:00.0 > /sys/bus/pci/devices/0000:44:00.0/driver/unbind

PREREQS=""
DEVS=“0000:43:00.0 0000:44:00.0 0000:65:00.0 0000:65:00.1”
for DEV in $DEVS;
do echo “vfio-pci” > /sys/bus/pci/devices/$DEV/driver_override
done

modprobe -i vfio-pci

So i am getting to installing windows.
When i reach the part where i can select the drives where Windows Installs, i load the VFIO driver, since both drives are raw images with virtion (not sata), i get a message windows cant be installed on this disks

Machine Firmware is BIOS and Chipset is i440FX.

Curiosly, i had managed to install windows short before this try, but i didnt had all the stuff passed through and ready like i have it now.

Any ideeas what the hell is wrong now ?
It seems every step of the way there is something.

0 Likes

#25

I suspect the permission are off for the img files that act as drives as seen by Windows. Anyone any ideea what those permissions might be ?

0 Likes

#26

sorry, for some reasons i have not bin getting notifications for this post. So i will put it on watch :smiley:
I’m glad you are having success.

The reason you are having issues installing windows is that it requires UEFI (not bios), In your first post you referenced the pop os guide, Go to it and search for OVMF.
Also everyone seems to use the Q35 chipset, but i have yet to understand why.

And ty for the posts about vfio-bind, i know @sgtawesomesauce did something similar, But few have posted about it. And i would love to learn more.

0 Likes

#27

I attempted something similar a while back, but it wasn’t stable enough for me to be satisfied with using it.

I just wound up getting another GPU.

1 Like

#28

Cant say ive been having succes. I deleted the base network that showed up with virt-manager, and cant seem to be able to create a new one. I am seeing no interfaces here which i could use to create a virtual network.
I created a bridge with the GUI network manager from the OS. Using that in the windows VM doesnt give me internet.
Setting up windows 10 works, with UEFi q35 machine. however, the only way to “maybe” use host pci devices, is to add them up once windows is booted and im in.
Starting the VM with added pci devices doesnt boot it. I need to remove them, and add them once im in windows.
Wat da faq ! Im getting tired of this.

0 Likes