Hello!
this day I was experimenting with an Asrock Arc Pro B60 and I wanted to try SR-IOV GPU virtualization in proxmox (9.2.10)…
newly installed machine, nothing special. Of course sr-iov enabled in BIOS and above 4g decoding. disabled CSM. In the first place, it everything seemed as expected, I could enable 4 virtual functions and create a mapping in Proxmox, assiged the mapping to the vm an booted up. Unfornately I ended up with error 43 in the VM. After a bit trial and error I decided I would try with the whole GPU in the vm first. So I assigend the GPU to the VM and bootet, my virtual functions disappeared, but the card worked correctly in Windows guest.
I tried to enable SR IOV again, whoch failed. So i rebooted the host and afterwards tried again, but this time I got a differen error:
[ 288.451761] xe 0000:04:00.0: [drm] PF: Tile0: GT0: VF1..VF4 provisioned with 536870912 (512 MiB) GGTT
[ 288.453980] xe 0000:04:00.0: [drm] PF: Tile0: GT0: VF1..VF4 provisioned with 6371147776 (5.93 GiB) LMEM
[ 288.454215] xe 0000:04:00.0: [drm] PF: Tile0: GT0: VF1..VF4 provisioned with 8192 GuC context IDs
[ 288.454439] xe 0000:04:00.0: [drm] PF: Tile0: GT0: VF1..VF4 provisioned with 32 GuC doorbell IDs
[ 288.454774] xe 0000:04:00.0: [drm] PF: Tile0: GT1: VF1..VF4 provisioned with 8192 GuC context IDs
[ 288.454972] xe 0000:04:00.0: [drm] PF: Tile0: GT1: VF1..VF4 provisioned with 32 GuC doorbell IDs
[ 288.454979] xe 0000:04:00.0: not enough MMIO resources for SR-IOV
[ 288.509740] xe 0000:04:00.0: [drm] PF: Failed to enable 4 VFs (-ENOMEM)
I cannot enable sr-iov anymore at all, although I am pretty sure I did not persist any settings.
Furthermore the card now seems to report “only“ 7 virtual functions while I am pretty sure! it where more in the first place (12 or 24?)… so loading the intel driver suite in windows seems to have changed sth about the cards behaviour? Is this possible?
And finally, I seems I am not even able anymore to assign the whole card to the vm. I mean, I can assign it but it renders the VM defunct. It does start without error but never seems to boot up, just allocating all ram and never finishind to start up.
in the host log I see
[ 492.255988] vfio-pci 0000:04:00.0: resetting
[ 492.363512] vfio-pci 0000:04:00.0: reset done
[ 492.406398] vfio-pci 0000:04:00.0: resetting
[ 492.507438] vfio-pci 0000:04:00.0: reset done
when I boot the vm and that is it. I tried disabling sr-iov in the BIOS which did not help. I cannot use the card anymore for virtualization as it seems. How is that even possbile? Does the intel driver “silently“ update firmware or change any settings of the card?
fyi: this is a supermicro x10-sri-f (yes it is an old board, but in general it should support the relevant functions).