Intel i915 sr-iov mode for Flex 170 + Proxmox PVE Kernel 6.5

Hey Gang,

I just watched the video from my sr-iov vm in 4k, and currently making this post from said vm.

I do have a half decent 4k client solution for your vgpu virtual machines.
I use a combination of 3 solutions.

  1. MolotovCherry virtual-display-rs on github
    This is for creating a ‘virtual display’ when rdp connects this happens, any other client has to create a virtual display. This one has a nice gui and persists reboots. There are some performance issues that i will get into later. People with dedicated GPUs get the best results with ‘dummy plugs’(thats gonna be important later). There are lots of different solutions.
  2. lizardbytes sunshine on github
    This one alot of people are probably familiar with, but in short this is a screen capture server designed for gaming, so it works for everything else pretty good.
    3.moonlight-qt on github
    Moonlight, said high performance client for accessing your vms remotely. With intel igpu borderless windowed mode is key.

I can get 4k video play back with this and not skip a beat as long as the video is not a 4k60fps video. Any more than 40fps in 4k is just too much for this setup. Some 4k gameplay is possible, just don’t have the highest expectations with 5 out of 80 execution units of an igpu.

One tradeoff with this setup is the hardware acceleration for sunshine/moonlight doesn’t work. This could be due feature regression in the intel drivers, or feature add with newer versions of intel drivers that require changes to moonlight. idk. I use software h264 for lowest cpu consumption.

These performance limitations I suspect are to be blamed on the virtual display, which intel has a solution already designed and ready to go, somebody just has to test it out.

on github intel Display-Virtualization-for-Windows-OS
This Github describes passing physical displayports on your device to vms, via customization to qemu.

intel also provides a driver :iotg-display-virtualization-drivers.html
They also provide this driver for usage with windows vms.

A whitepaper by a company called DFI about SRIOV led me down this rabbit hole and I am trying to figure out how to make a miracast display work on a vm.

I wish I could have included the links :frowning:

2 Likes

I’m not sure which if any reseller procurement is going through, but it is Supermicro. I just gave the list of specs I wanted and sent it off. I asked for a couple tower versions too and they came back asking if a different variant of said tower was acceptable so supermicro engineering is willing to do it, or so it seems.

Can you please expand of what you mean by not fully backed on gen 10/11 for sr-iov. I have two free hosts i want to try this on, but they are gen 11, what should I expect?

Also amazing post and great work.

Try the i915 backports and report back lmk

did someone allready test the arc 750 or 770 teased in the video?
over here its 333Bucks and for a new build would be great to have and better than the need to have patched/custom nvidia drivers

Is this at all possible with an AMD rx6400? I’m only looking to split it for 2 vms. I have virgl GPU working but of course it only works when using virt-viewer and spice. I would much rather have dedicated GPU for vm so i can use the acceleration even when I’m not using a spice client.

I’ve attempted to build the i915 drivers against ProxMox’s 6.5.x PVE kernel for an Intel A380, but w/ no love.

I have the A380 slotted into an older AsRock X99X LGA-2011 socket board w/ a Xeon E5-2660 v3 installed, but the latest BIOS does have the option for SR-IOV. Just looking to split the GPU for more than just Plex duties. Does it take more than just BIOS support to make proper SR-IOV work end-to-end? I hate to e-waste a running system for just 1-2 features.

1 Like

Hello All,
so I got an Intel Flex 140 to try and test in Proxmox.
and I only have AMD Epyc servers to use this in…
compiling the driver went all fine, and from the dmesg output it loads OK.
I am unable to enable the desired virtual functions by entering a number to sriov_numvfs.
Maybe I am getting to the relevant folder incorrectly, not through the right bridge…

lspci | grep Flex 
87:00.0 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
8a:00.0 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
dmesg | grep i915
[    0.000000] Command line: initrd=\EFI\proxmox\6.5.13-3-pve\initrd.img-6.5.13-3-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs initcall_blacklist=acpi_cpufreq_init amd_pstate=active amd_pstate.shared_mem=1 i915.enable_guc=3 i915.max_vfs=3
[    0.212536] Kernel command line: initrd=\EFI\proxmox\6.5.13-3-pve\initrd.img-6.5.13-3-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs initcall_blacklist=acpi_cpufreq_init amd_pstate=active amd_pstate.shared_mem=1 i915.enable_guc=3 i915.max_vfs=3
[   46.212175] i915 0000:87:00.0: Running in SR-IOV PF mode
[   46.212613] i915 0000:87:00.0: [drm] GT count: 1, enabled: 1
[   46.213448] i915 0000:87:00.0: [drm] VT-d active for gfx access
[   46.213900] i915 0000:87:00.0: [drm] Using Transparent Hugepages
[   46.214348] i915 0000:87:00.0: [drm] Local memory IO size: 0x0000000140000000
[   46.214377] i915 0000:87:00.0: [drm] Local memory available: 0x000000013cc00000
[   46.218172] i915 0000:87:00.0: [drm] GT0: HuC firmware i915/dg2_huc_7.10.14_gsc.bin (7.10.14) is recommended, but only i915/dg2_huc_7.10.14_gsc.bin (7.10.3) was found
[   46.218230] i915 0000:87:00.0: [drm] GT0: Consider updating your linux-firmware pkg or downloading from https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915
[   46.226597] i915 0000:87:00.0: [drm] GT0: GuC firmware i915/dg2_guc_70.19.2.bin version 70.20.0
[   46.226980] i915 0000:87:00.0: [drm] GT0: HuC firmware i915/dg2_huc_7.10.14_gsc.bin version 7.10.3
[   46.237504] i915 0000:87:00.0: [drm] GT0: GUC: submission enabled
[   46.237507] i915 0000:87:00.0: [drm] GT0: GUC: SLPC enabled
[   46.237729] i915 0000:87:00.0: [drm] GT0: GUC: RC enabled
[   46.272826] i915 0000:87:00.0: GT0: local0 bcs'0.0 clear bandwidth:106100 MB/s
[   46.285145] i915 0000:87:00.0: GT0: local0 bcs'0.0 swap bandwidth:2362 MB/s
[   46.285569] i915 0000:87:00.0: 3 VFs could be associated with this PF
[   46.286344] [drm] Initialized i915 1.6.0 20201103 for 0000:87:00.0 on minor 1
[   46.320844] i915 0000:87:00.0: SPI access overridden by jumper
[   46.341173] i915 0000:8a:00.0: Running in SR-IOV PF mode
[   46.341667] i915 0000:8a:00.0: [drm] GT count: 1, enabled: 1
[   46.342601] i915 0000:8a:00.0: [drm] VT-d active for gfx access
[   46.342972] i915 0000:8a:00.0: [drm] Using Transparent Hugepages
[   46.343335] i915 0000:8a:00.0: [drm] Local memory IO size: 0x0000000140000000
[   46.343637] i915 0000:8a:00.0: [drm] Local memory available: 0x000000013cc00000
[   46.345429] i915 0000:8a:00.0: [drm] GT0: HuC firmware i915/dg2_huc_7.10.14_gsc.bin (7.10.14) is recommended, but only i915/dg2_huc_7.10.14_gsc.bin (7.10.3) was found
[   46.346015] i915 0000:8a:00.0: [drm] GT0: Consider updating your linux-firmware pkg or downloading from https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/tree/i915
[   46.353163] i915 0000:8a:00.0: [drm] GT0: GuC firmware i915/dg2_guc_70.19.2.bin version 70.20.0
[   46.353487] i915 0000:8a:00.0: [drm] GT0: HuC firmware i915/dg2_huc_7.10.14_gsc.bin version 7.10.3
[   46.365461] i915 0000:8a:00.0: [drm] GT0: GUC: submission enabled
[   46.365781] i915 0000:8a:00.0: [drm] GT0: GUC: SLPC enabled
[   46.366274] i915 0000:8a:00.0: [drm] GT0: GUC: RC enabled
[   46.396210] i915 0000:8a:00.0: GT0: local0 bcs'0.0 clear bandwidth:105995 MB/s
[   46.409447] i915 0000:8a:00.0: GT0: local0 bcs'0.0 swap bandwidth:2343 MB/s
[   46.409789] i915 0000:8a:00.0: 3 VFs could be associated with this PF
[   46.410556] [drm] Initialized i915 1.6.0 20201103 for 0000:8a:00.0 on minor 2
[   46.411206] i915 0000:8a:00.0: SPI access overridden by jumper
[   46.447512] Creating 4 MTD partitions on "i915.spi.34560":
[   46.448217] 0x000000000000-0x000000001000 : "i915.spi.34560.DESCRIPTOR"
[   46.450999] 0x000000001000-0x0000005f0000 : "i915.spi.34560.GSC"
[   46.453700] 0x0000005f0000-0x0000007f0000 : "i915.spi.34560.OptionROM"
[   46.456084] 0x0000007f0000-0x000000800000 : "i915.spi.34560.DAM"
[   46.460264] mei i915.mei-gscfi.34560-46e0c1fb-a546-414f-9170-b7f46d57b4ad: Could not read FW version ret = -19
[   46.460791] mei i915.mei-gscfi.34560-46e0c1fb-a546-414f-9170-b7f46d57b4ad: FW version command failed -5
[   46.461127] Creating 4 MTD partitions on "i915.spi.35328":
[   46.461697] 0x000000000000-0x000000001000 : "i915.spi.35328.DESCRIPTOR"
[   46.463348] mei i915.mei-gscfi.35328-46e0c1fb-a546-414f-9170-b7f46d57b4ad: Could not read FW version ret = -19
[   46.463783] mei i915.mei-gscfi.35328-46e0c1fb-a546-414f-9170-b7f46d57b4ad: FW version command failed -5
[   46.464069] 0x000000001000-0x0000005f0000 : "i915.spi.35328.GSC"
[   46.466585] 0x0000005f0000-0x0000007f0000 : "i915.spi.35328.OptionROM"
[   46.469138] 0x0000007f0000-0x000000800000 : "i915.spi.35328.DAM"
[   47.479826] i915 0000:87:00.0: [drm] GT0: HuC: authenticated!
[   47.480477] mei_pxp i915.mei-gsc.34560-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:87:00.0 (ops i915_pxp_tee_component_ops [i915])
[   47.510479] i915 0000:8a:00.0: [drm] GT0: HuC: authenticated!
[   47.510980] mei_pxp i915.mei-gsc.35328-fbf6fcf1-96cf-4e2e-a6a6-1bab8cbe36b1: bound 0000:8a:00.0 (ops i915_pxp_tee_component_ops [i915])
find /sys/devices -name 'sriov_numvfs'
/sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/0000:84:18.0/0000:88:00.0/0000:89:01.0/0000:8a:00.0/sriov_numvfs
/sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/0000:84:08.0/0000:85:00.0/0000:86:01.0/0000:87:00.0/sriov_numvfs
echo 6 > /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/0000:84:18.0/0000:88:00.0/0000:89:01.0/0000:8a:00.0/sriov_numvfs
-bash: echo: write error: No such file or directory

any thoughts what I am doing wrong here?
Thanks

check dmesg output after this to see if there was any output on failure to create number of functions requested?

also cat /proc/cmdline for us?

thanks Wendell!!!

root@epyc:/# cat /proc/cmdline
initrd=\EFI\proxmox\6.5.13-3-pve\initrd.img-6.5.13-3-pve root=ZFS=rpool/ROOT/pve-1 boot=zfs initcall_blacklist=acpi_cpufreq_init amd_pstate=active amd_pstate.shared_mem=1 i915.enable_guc=3 i915.max_vfs=3

nothing in dmesg

i915.max_vfs=3
this might mean you need to echo 3 into that not 6?

sorry, yes tried that as well, same error, nothing in dmesg, just pasted here the line where I tried ‘6’

 cat /sys/devices/pci0000:80/0000:80:01.1/0000:81:00.0/0000:82:00.0/0000:83:00.0/0000:84:08.0/0000:85:00.0/0000:86:01.0/0000:87:00.0/sriov_numvfs
0

Maybesee if i915 - kernel/git/firmware/linux-firmware.git - Repository of firmware blobs for use with the Linux kernel has a newer firmware? Im afk

Sir, you are a legend!

I indeed pulled the most recent firmware blobs from the link you posted above, and also symlinked the latest version to be a specific version the card was looking for.
If you go back to my dmesg output above, you will note that the card loads the symbolic link with a version and outputs the version the blobs is.
so dg2_guc_70.19.2.bin was actually 70.20.0 and dg2_huc_7.10.14_gsc.bin actually 7.10.3, i.e. older than the card was looking for.

A search for this file turned up a github repo that seem to hold all versions of these blobs, at https://github.com/intel-gpu/intel-gpu-firmware/tree/main/firmware

Picked the versions the card was looking for from there, rebooted and voila all is good, I could enable the desired number virtual functions.

Of note that i915.max_vfs=number defines the max. I could echo 1,2,3 when set this to 6. I could not enable more than 3 if the max was set to 3.

So this is how is populated now: 6x 1GB vfs + 3 x 2GB vfs for a total of 9

lspci | grep Flex
87:00.0 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
87:00.1 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
87:00.2 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
87:00.3 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
87:00.4 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
87:00.5 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
87:00.6 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
8a:00.0 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
8a:00.1 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
8a:00.2 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)
8a:00.3 Display controller: Intel Corporation Data Center GPU Flex 140 (rev 05)

this is the dmesg output after I enabled the VFs

[  597.224623] i915 0000:8a:00.0: [drm] VF BAR2 resized to 8192M
[  597.332254] pci 0000:8a:00.1: [8086:56c1] type 00 class 0x038000
[  597.332881] pci 0000:8a:00.1: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  597.333554] pci 0000:8a:00.1: Adding to iommu group 99
[  597.334254] BACKPORTED INTEL VSEC PCI PROBE
[  597.334594] intel_vsec 0000:8a:00.1: enabling device (0000 -> 0002)
[  597.335005] i915 0000:8a:00.1: enabling device (0000 -> 0002)
[  597.335379] i915 0000:8a:00.1: Running in SR-IOV VF mode
[  597.336137] i915 0000:8a:00.1: GuC interface version 0.1.8.2
[  597.336885] i915 0000:8a:00.1: [drm] GT count: 1, enabled: 1
[  597.337369] i915 0000:8a:00.1: [drm] VT-d active for gfx access
[  597.337702] i915 0000:8a:00.1: [drm] Using Transparent Hugepages
[  597.338104] i915 0000:8a:00.1: [drm] Local memory IO size: 0x0000000066000000
[  597.338384] i915 0000:8a:00.1: [drm] Local memory available: 0x0000000066000000
[  597.339197] i915 0000:8a:00.1: GuC interface version 0.1.8.2
[  597.339870] i915 0000:8a:00.1: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  597.340158] i915 0000:8a:00.1: HuC firmware PRELOADED
[  597.371613] i915 0000:8a:00.1: GT0: local0 bcs'0.0 clear bandwidth:105891 MB/s
[  597.382636] i915 0000:8a:00.1: GT0: local0 bcs'0.0 swap bandwidth:2348 MB/s
[  597.383245] [drm] Initialized i915 1.6.0 20201103 for 0000:8a:00.1 on minor 3
[  597.388899] BACKPORTED INTEL VSEC REGISTER
[  597.389450] pci 0000:8a:00.2: [8086:56c1] type 00 class 0x038000
[  597.389821] pci 0000:8a:00.2: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  597.390425] pci 0000:8a:00.2: Adding to iommu group 100
[  597.391230] BACKPORTED INTEL VSEC PCI PROBE
[  597.391575] intel_vsec 0000:8a:00.2: enabling device (0000 -> 0002)
[  597.391991] i915 0000:8a:00.2: enabling device (0000 -> 0002)
[  597.392356] i915 0000:8a:00.2: Running in SR-IOV VF mode
[  597.392834] i915 0000:8a:00.2: GuC interface version 0.1.8.2
[  597.393475] i915 0000:8a:00.2: [drm] GT count: 1, enabled: 1
[  597.393946] i915 0000:8a:00.2: [drm] VT-d active for gfx access
[  597.394301] i915 0000:8a:00.2: [drm] Using Transparent Hugepages
[  597.394650] i915 0000:8a:00.2: [drm] Local memory IO size: 0x0000000066000000
[  597.394965] i915 0000:8a:00.2: [drm] Local memory available: 0x0000000066000000
[  597.395826] i915 0000:8a:00.2: GuC interface version 0.1.8.2
[  597.396614] i915 0000:8a:00.2: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  597.396951] i915 0000:8a:00.2: HuC firmware PRELOADED
[  597.462618] i915 0000:8a:00.2: GT0: local0 bcs'0.0 clear bandwidth:27310 MB/s
[  597.472041] i915 0000:8a:00.2: GT0: local0 bcs'0.0 swap bandwidth:2376 MB/s
[  597.472806] [drm] Initialized i915 1.6.0 20201103 for 0000:8a:00.2 on minor 4
[  597.480281] BACKPORTED INTEL VSEC REGISTER
[  597.481194] pci 0000:8a:00.3: [8086:56c1] type 00 class 0x038000
[  597.481608] pci 0000:8a:00.3: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  597.482206] pci 0000:8a:00.3: Adding to iommu group 101
[  597.482955] BACKPORTED INTEL VSEC PCI PROBE
[  597.483289] intel_vsec 0000:8a:00.3: enabling device (0000 -> 0002)
[  597.483692] i915 0000:8a:00.3: enabling device (0000 -> 0002)
[  597.484040] i915 0000:8a:00.3: Running in SR-IOV VF mode
[  597.484509] i915 0000:8a:00.3: GuC interface version 0.1.8.2
[  597.485152] i915 0000:8a:00.3: [drm] GT count: 1, enabled: 1
[  597.485605] i915 0000:8a:00.3: [drm] VT-d active for gfx access
[  597.485944] i915 0000:8a:00.3: [drm] Using Transparent Hugepages
[  597.486273] i915 0000:8a:00.3: [drm] Local memory IO size: 0x0000000066000000
[  597.486562] i915 0000:8a:00.3: [drm] Local memory available: 0x0000000066000000
[  597.487354] i915 0000:8a:00.3: GuC interface version 0.1.8.2
[  597.488012] i915 0000:8a:00.3: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  597.488308] i915 0000:8a:00.3: HuC firmware PRELOADED
[  597.558540] i915 0000:8a:00.3: GT0: local0 bcs'0.0 clear bandwidth:27310 MB/s
[  597.570464] i915 0000:8a:00.3: GT0: local0 bcs'0.0 swap bandwidth:2359 MB/s
[  597.571192] [drm] Initialized i915 1.6.0 20201103 for 0000:8a:00.3 on minor 5
[  597.576581] BACKPORTED INTEL VSEC REGISTER
[  597.577498] i915 0000:8a:00.0: Enabled 3 VFs
[  616.232266] i915 0000:87:00.0: [drm] VF BAR2 resized to 8192M
[  616.339780] pci 0000:87:00.1: [8086:56c1] type 00 class 0x038000
[  616.340459] pci 0000:87:00.1: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  616.341098] pci 0000:87:00.1: Adding to iommu group 102
[  616.341787] BACKPORTED INTEL VSEC PCI PROBE
[  616.342114] intel_vsec 0000:87:00.1: enabling device (0000 -> 0002)
[  616.342517] i915 0000:87:00.1: enabling device (0000 -> 0002)
[  616.342880] i915 0000:87:00.1: Running in SR-IOV VF mode
[  616.343623] i915 0000:87:00.1: GuC interface version 0.1.8.2
[  616.344455] i915 0000:87:00.1: [drm] GT count: 1, enabled: 1
[  616.344942] i915 0000:87:00.1: [drm] VT-d active for gfx access
[  616.345276] i915 0000:87:00.1: [drm] Using Transparent Hugepages
[  616.345600] i915 0000:87:00.1: [drm] Local memory IO size: 0x0000000033000000
[  616.345885] i915 0000:87:00.1: [drm] Local memory available: 0x0000000033000000
[  616.346699] i915 0000:87:00.1: GuC interface version 0.1.8.2
[  616.347368] i915 0000:87:00.1: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  616.347650] i915 0000:87:00.1: HuC firmware PRELOADED
[  616.378964] i915 0000:87:00.1: GT0: local0 bcs'0.0 clear bandwidth:105856 MB/s
[  616.390171] i915 0000:87:00.1: GT0: local0 bcs'0.0 swap bandwidth:2353 MB/s
[  616.390744] [drm] Initialized i915 1.6.0 20201103 for 0000:87:00.1 on minor 6
[  616.396208] BACKPORTED INTEL VSEC REGISTER
[  616.396794] pci 0000:87:00.2: [8086:56c1] type 00 class 0x038000
[  616.397107] pci 0000:87:00.2: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  616.397606] pci 0000:87:00.2: Adding to iommu group 103
[  616.398196] BACKPORTED INTEL VSEC PCI PROBE
[  616.398483] intel_vsec 0000:87:00.2: enabling device (0000 -> 0002)
[  616.398864] i915 0000:87:00.2: enabling device (0000 -> 0002)
[  616.399172] i915 0000:87:00.2: Running in SR-IOV VF mode
[  616.399655] i915 0000:87:00.2: GuC interface version 0.1.8.2
[  616.400321] i915 0000:87:00.2: [drm] GT count: 1, enabled: 1
[  616.400746] i915 0000:87:00.2: [drm] VT-d active for gfx access
[  616.401062] i915 0000:87:00.2: [drm] Using Transparent Hugepages
[  616.401375] i915 0000:87:00.2: [drm] Local memory IO size: 0x0000000033000000
[  616.401651] i915 0000:87:00.2: [drm] Local memory available: 0x0000000033000000
[  616.402434] i915 0000:87:00.2: GuC interface version 0.1.8.2
[  616.403102] i915 0000:87:00.2: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  616.403382] i915 0000:87:00.2: HuC firmware PRELOADED
[  616.449721] i915 0000:87:00.2: GT0: local0 bcs'0.0 clear bandwidth:27307 MB/s
[  616.460044] i915 0000:87:00.2: GT0: local0 bcs'0.0 swap bandwidth:2306 MB/s
[  616.460876] [drm] Initialized i915 1.6.0 20201103 for 0000:87:00.2 on minor 7
[  616.467398] BACKPORTED INTEL VSEC REGISTER
[  616.468185] pci 0000:87:00.3: [8086:56c1] type 00 class 0x038000
[  616.468609] pci 0000:87:00.3: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  616.469215] pci 0000:87:00.3: Adding to iommu group 104
[  616.469875] BACKPORTED INTEL VSEC PCI PROBE
[  616.470215] intel_vsec 0000:87:00.3: enabling device (0000 -> 0002)
[  616.470628] i915 0000:87:00.3: enabling device (0000 -> 0002)
[  616.470973] i915 0000:87:00.3: Running in SR-IOV VF mode
[  616.471426] i915 0000:87:00.3: GuC interface version 0.1.8.2
[  616.472067] i915 0000:87:00.3: [drm] GT count: 1, enabled: 1
[  616.472526] i915 0000:87:00.3: [drm] VT-d active for gfx access
[  616.472862] i915 0000:87:00.3: [drm] Using Transparent Hugepages
[  616.473195] i915 0000:87:00.3: [drm] Local memory IO size: 0x0000000033000000
[  616.473486] i915 0000:87:00.3: [drm] Local memory available: 0x0000000033000000
[  616.474246] i915 0000:87:00.3: GuC interface version 0.1.8.2
[  616.474889] i915 0000:87:00.3: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  616.475170] i915 0000:87:00.3: HuC firmware PRELOADED
[  616.522155] i915 0000:87:00.3: GT0: local0 bcs'0.0 clear bandwidth:27310 MB/s
[  616.533347] i915 0000:87:00.3: GT0: local0 bcs'0.0 swap bandwidth:2365 MB/s
[  616.533899] [drm] Initialized i915 1.6.0 20201103 for 0000:87:00.3 on minor 8
[  616.540122] BACKPORTED INTEL VSEC REGISTER
[  616.540711] pci 0000:87:00.4: [8086:56c1] type 00 class 0x038000
[  616.541097] pci 0000:87:00.4: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  616.541629] pci 0000:87:00.4: Adding to iommu group 105
[  616.542219] BACKPORTED INTEL VSEC PCI PROBE
[  616.542526] intel_vsec 0000:87:00.4: enabling device (0000 -> 0002)
[  616.542910] i915 0000:87:00.4: enabling device (0000 -> 0002)
[  616.543231] i915 0000:87:00.4: Running in SR-IOV VF mode
[  616.543651] i915 0000:87:00.4: GuC interface version 0.1.8.2
[  616.544203] i915 0000:87:00.4: [drm] GT count: 1, enabled: 1
[  616.544637] i915 0000:87:00.4: [drm] VT-d active for gfx access
[  616.544956] i915 0000:87:00.4: [drm] Using Transparent Hugepages
[  616.545268] i915 0000:87:00.4: [drm] Local memory IO size: 0x0000000033000000
[  616.545545] i915 0000:87:00.4: [drm] Local memory available: 0x0000000033000000
[  616.546319] i915 0000:87:00.4: GuC interface version 0.1.8.2
[  616.546969] i915 0000:87:00.4: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  616.547247] i915 0000:87:00.4: HuC firmware PRELOADED
[  616.594109] i915 0000:87:00.4: GT0: local0 bcs'0.0 clear bandwidth:27307 MB/s
[  616.603512] i915 0000:87:00.4: GT0: local0 bcs'0.0 swap bandwidth:2393 MB/s
[  616.604310] [drm] Initialized i915 1.6.0 20201103 for 0000:87:00.4 on minor 9
[  616.611503] BACKPORTED INTEL VSEC REGISTER
[  616.612322] pci 0000:87:00.5: [8086:56c1] type 00 class 0x038000
[  616.612713] pci 0000:87:00.5: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  616.613260] pci 0000:87:00.5: Adding to iommu group 106
[  616.613908] BACKPORTED INTEL VSEC PCI PROBE
[  616.614227] intel_vsec 0000:87:00.5: enabling device (0000 -> 0002)
[  616.614635] i915 0000:87:00.5: enabling device (0000 -> 0002)
[  616.614984] i915 0000:87:00.5: Running in SR-IOV VF mode
[  616.615447] i915 0000:87:00.5: GuC interface version 0.1.8.2
[  616.616003] i915 0000:87:00.5: [drm] GT count: 1, enabled: 1
[  616.616456] i915 0000:87:00.5: [drm] VT-d active for gfx access
[  616.616787] i915 0000:87:00.5: [drm] Using Transparent Hugepages
[  616.617111] i915 0000:87:00.5: [drm] Local memory IO size: 0x0000000033000000
[  616.617406] i915 0000:87:00.5: [drm] Local memory available: 0x0000000033000000
[  616.618139] i915 0000:87:00.5: GuC interface version 0.1.8.2
[  616.618788] i915 0000:87:00.5: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  616.619088] i915 0000:87:00.5: HuC firmware PRELOADED
[  616.669470] i915 0000:87:00.5: GT0: local0 bcs'0.0 clear bandwidth:27310 MB/s
[  616.682135] i915 0000:87:00.5: GT0: local0 bcs'0.0 swap bandwidth:2366 MB/s
[  616.682768] [drm] Initialized i915 1.6.0 20201103 for 0000:87:00.5 on minor 10
[  616.687235] BACKPORTED INTEL VSEC REGISTER
[  616.687863] pci 0000:87:00.6: [8086:56c1] type 00 class 0x038000
[  616.688271] pci 0000:87:00.6: ASPM: overriding L1 acceptable latency from 0x0 to 0x7
[  616.688793] pci 0000:87:00.6: Adding to iommu group 107
[  616.689371] BACKPORTED INTEL VSEC PCI PROBE
[  616.689676] intel_vsec 0000:87:00.6: enabling device (0000 -> 0002)
[  616.690077] i915 0000:87:00.6: enabling device (0000 -> 0002)
[  616.690400] i915 0000:87:00.6: Running in SR-IOV VF mode
[  616.690834] i915 0000:87:00.6: GuC interface version 0.1.8.2
[  616.691385] i915 0000:87:00.6: [drm] GT count: 1, enabled: 1
[  616.691862] i915 0000:87:00.6: [drm] VT-d active for gfx access
[  616.692189] i915 0000:87:00.6: [drm] Using Transparent Hugepages
[  616.692509] i915 0000:87:00.6: [drm] Local memory IO size: 0x0000000033000000
[  616.692791] i915 0000:87:00.6: [drm] Local memory available: 0x0000000033000000
[  616.693572] i915 0000:87:00.6: GuC interface version 0.1.8.2
[  616.694239] i915 0000:87:00.6: GuC firmware PRELOADED version 0.0 submission:SR-IOV VF
[  616.694523] i915 0000:87:00.6: HuC firmware PRELOADED
[  616.740253] i915 0000:87:00.6: GT0: local0 bcs'0.0 clear bandwidth:27310 MB/s
[  616.749484] i915 0000:87:00.6: GT0: local0 bcs'0.0 swap bandwidth:2365 MB/s
[  616.750285] [drm] Initialized i915 1.6.0 20201103 for 0000:87:00.6 on minor 11
[  616.757945] BACKPORTED INTEL VSEC REGISTER
[  616.758912] i915 0000:87:00.0: Enabled 6 VFs

@wendell have you tried these with a pure i915 integrated GPU?

I asks because i note you are able to get these working with the CPU not set to host!?!

when i set the CPU to x86-64-v2-AES with the strongtz drivers i get code 43 100% of them so I am thinking of trying these instead…

Yes the igpu is a bit sketchier but it can work. Did a video with the minisforum ms01

cool, trying that now, i didn’t watch the video as

  1. i have no clue what a minisforum ms01 is and why i would watch that
  2. i am not a millennial who can’t read more than a page of instructions :wink:

hehe, nice write up BTW

compile had no issues, yup got firmware issue the right place for people to get these fimwares is definitely from intel-gpu-firmware/firmware at main · intel-gpu/intel-gpu-firmware (github.com) the path given in dmesg doesn’t have them…

hmm thats not good… this is when the VM attempts to start…

[ 2595.031902] BUG: kernel NULL pointer dereference, address: 0000000000000188
[ 2595.031927] #PF: supervisor write access in kernel mode
[ 2595.031935] #PF: error_code(0x0002) - not-present page
[ 2595.031944] PGD 0 P4D 0 
[ 2595.031957] Oops: 0002 [#1] PREEMPT SMP NOPTI
[ 2595.031969] CPU: 14 PID: 17033 Comm: kworker/u32:3 Tainted: P     U     O       6.5.13-3-pve #1
[ 2595.031984] Hardware name: Intel(R) Client Systems NUC13ANHi7/NUC13ANBi7, BIOS ANRPL357.0026.2023.0314.1458 03/14/2023
[ 2595.031995] Workqueue: i915 __i915_vm_release [i915]
[ 2595.032468] RIP: 0010:_raw_spin_lock+0x13/0x60
[ 2595.032488] Code: 31 db c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 65 ff 05 3c 7a fb 5c 31 c0 ba 01 00 00 00 <f0> 0f b1 17 75 1b 31 c0 31 d2 31 c9 31 f6 31 ff 45 31 c0 45 31 c9
[ 2595.032505] RSP: 0018:ffffb450a74efcb0 EFLAGS: 00010246
[ 2595.032515] RAX: 0000000000000000 RBX: ffff96ed01d38cc0 RCX: 0000000000000000
[ 2595.032524] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000188
[ 2595.032532] RBP: ffffb450a74efcd0 R08: 0000000000000000 R09: 0000000000000000
[ 2595.032540] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 2595.032548] R13: 0000000000000188 R14: ffff96ed125a1000 R15: ffff96ed125a1000
[ 2595.032556] FS:  0000000000000000(0000) GS:ffff96fc57980000(0000) knlGS:0000000000000000
[ 2595.032567] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2595.032575] CR2: 0000000000000188 CR3: 0000000b95434000 CR4: 0000000000752ee0
[ 2595.032585] PKRU: 55555554
[ 2595.032590] Call Trace:
[ 2595.032597]  <TASK>
[ 2595.032605]  ? show_regs+0x6d/0x80
[ 2595.032621]  ? __die+0x24/0x80
[ 2595.032631]  ? page_fault_oops+0x176/0x500
[ 2595.032647]  ? do_user_addr_fault+0x31d/0x6a0
[ 2595.032660]  ? exc_page_fault+0x83/0x1b0
[ 2595.032670]  ? asm_exc_page_fault+0x27/0x30
[ 2595.032689]  ? _raw_spin_lock+0x13/0x60
[ 2595.032701]  ? px_release+0x28/0xe0 [i915]
[ 2595.033107]  free_px+0x7c/0xb0 [i915]
[ 2595.033506]  __gen8_ppgtt_cleanup+0x3db/0x410 [i915]
[ 2595.033867]  ? psi_task_switch+0xd3/0x240
[ 2595.033880]  ? raw_spin_rq_unlock+0x10/0x40
[ 2595.033891]  ? finish_task_switch.isra.0+0x85/0x2c0
[ 2595.033902]  ? __schedule+0x404/0x1440
[ 2595.033917]  gen8_ppgtt_cleanup+0x3a/0x60 [i915]
[ 2595.034275]  __i915_vm_release+0x1a/0x40 [i915]
[ 2595.034625]  process_one_work+0x23b/0x450
[ 2595.034638]  worker_thread+0x50/0x3f0
[ 2595.034648]  ? __pfx_worker_thread+0x10/0x10
[ 2595.034658]  kthread+0xef/0x120
[ 2595.034666]  ? __pfx_kthread+0x10/0x10
[ 2595.034672]  ret_from_fork+0x44/0x70
[ 2595.034682]  ? __pfx_kthread+0x10/0x10
[ 2595.034689]  ret_from_fork_asm+0x1b/0x30
[ 2595.034700]  </TASK>
[ 2595.034704] Modules linked in: vfio_pci vfio_pci_core vfio_iommu_type1 vfio iommufd rbd veth ceph libceph fscache netfs ebtable_filter ebtables ip_set ip6table_raw iptable_raw ip6table_filter ip6_tables iptable_filter bpfilter sctp ip6_udp_tunnel udp_tunnel nf_tables nvme_fabrics bonding tls qrtr softdog sunrpc nfnetlink_log nfnetlink binfmt_misc snd_hda_codec_hdmi intel_rapl_msr snd_hda_codec_realtek intel_rapl_common snd_hda_codec_generic intel_uncore_frequency ledtrig_audio intel_uncore_frequency_common snd_sof_pci_intel_tgl snd_sof_intel_hda_common soundwire_intel snd_sof_intel_hda_mlink soundwire_cadence x86_pkg_temp_thermal snd_sof_intel_hda intel_powerclamp snd_sof_pci snd_sof_xtensa_dsp snd_sof snd_sof_utils snd_soc_hdac_hda coretemp snd_hda_ext_core snd_soc_acpi_intel_match snd_soc_acpi kvm_intel soundwire_generic_allocation iwlmvm soundwire_bus kvm snd_soc_core i915(O) mac80211 snd_compress ac97_bus irqbypass snd_pcm_dmaengine crct10dif_pclmul polyval_clmulni snd_hda_intel libarc4 polyval_generic
[ 2595.034834]  snd_intel_dspcfg btusb ghash_clmulni_intel snd_intel_sdw_acpi btrtl snd_hda_codec sha256_ssse3 btbcm i915_compat(O) sha1_ssse3 snd_hda_core ov13858 btintel drm_display_helper aesni_intel snd_hwdep v4l2_fwnode btmtk v4l2_async cec snd_pcm pmt_telemetry(O) crypto_simd bluetooth iwlwifi mei_hdcp(O) videodev mei_pxp(O) pmt_class(O) snd_timer cmdlinepart rc_core cryptd ucsi_acpi mei_me(O) spi_nor snd drm_kms_helper typec_ucsi rapl ecdh_generic joydev input_leds cfg80211 pcspkr wmi_bmof i2c_algo_bit soundcore intel_cstate mtd ee1004 ecc mei(O) typec intel_vsec(O) mc acpi_pad acpi_tad mac_hid zfs(PO) spl(O) vhost_net vhost vhost_iotlb tap thunderbolt_net msr drm efi_pstore dmi_sysfs ip_tables x_tables autofs4 btrfs blake2b_generic xor uas usb_storage raid6_pq simplefb hid_generic usbkbd usbmouse usbhid hid dm_thin_pool dm_persistent_data dm_bio_prison dm_bufio libcrc32c xhci_pci nvme intel_lpss_pci xhci_pci_renesas spi_intel_pci ahci i2c_i801 intel_lpss nvme_core video thunderbolt crc32_pclmul igc xhci_hcd
[ 2595.035055]  spi_intel libahci i2c_smbus idma64 nvme_common wmi pinctrl_tigerlake
[ 2595.035114] CR2: 0000000000000188
[ 2595.035121] ---[ end trace 0000000000000000 ]---
[ 2595.272909] RIP: 0010:_raw_spin_lock+0x13/0x60
[ 2595.272933] Code: 31 db c3 cc cc cc cc 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 0f 1f 44 00 00 65 ff 05 3c 7a fb 5c 31 c0 ba 01 00 00 00 <f0> 0f b1 17 75 1b 31 c0 31 d2 31 c9 31 f6 31 ff 45 31 c0 45 31 c9
[ 2595.272940] RSP: 0018:ffffb450a74efcb0 EFLAGS: 00010246
[ 2595.272944] RAX: 0000000000000000 RBX: ffff96ed01d38cc0 RCX: 0000000000000000
[ 2595.272947] RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000000000000188
[ 2595.272950] RBP: ffffb450a74efcd0 R08: 0000000000000000 R09: 0000000000000000
[ 2595.272952] R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
[ 2595.272955] R13: 0000000000000188 R14: ffff96ed125a1000 R15: ffff96ed125a1000
[ 2595.272958] FS:  0000000000000000(0000) GS:ffff96fc57980000(0000) knlGS:0000000000000000
[ 2595.272962] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 2595.272964] CR2: 0000000000000188 CR3: 0000000113582000 CR4: 0000000000752ee0
[ 2595.272967] PKRU: 55555554
[ 2595.272971] note: kworker/u32:3[17033] exited with irqs disabled
[ 2595.272998] note: kworker/u32:3[17033] exited with preempt_count 1

double check bios settings? might try rebar on/off (it should not be off) and sr-iov should be on. vt-d should be on. confirm bios looks okay? you might also, strange as this sounds, disable thunderbolt if you have the option.

You might also have to use one of the other git repos that has the slightly-patched version of the i915 driver.