Proxmox 9.0 + Intel B50 SR-IOV -- finally its almost here! Early Adopter's Guide

[IroesStrongarm]

Just got a test system setup just for testing the b50 for now until it’s fully ready for primetime and to be deployed in my home production server. This is my first time dealing with SR-IOV. I noticed that every time I reboot the host I have to redo the echo 4 > sriov_numvfs command.

I got a win11 VM setup, went through all the steps and did get a 4Gb VRAM machine up and running and tested.

Oddly, in GPU-Z it doesn’t show any PCI link speed for the Bus Interface. Not sure if this is because I’m running it as a portable app and didn’t fully install it. I couldn’t hit the question mark next to it and run the render test, but my screenshot clearly shows it working well in Windows.

I also noticed that GPU-Z states rebar enabled but the Intel driver software claims it isn’t. This is likely a reporting bug in the beta but wanted to mention it.

Tried getting this to work in an Ubuntu server VM. Didn’t work right but guessing that’s because I was on kernel 6.8 instead of 6.17. Will have to play around with it more.

[wendell]

ah so I think your assumption is the firmware blobs are all that there is. that isn’t so. you need to let the windows driver update yet more firmware that’s not runtime blobs. and make sure sr iov is on in bios.

[wendell]

@IroesStrongarm

. I noticed that every time I reboot the host I have to redo the echo 4 > sriov_numvfs

oops, I created a systemd service for this. will add it to the howto

[ynfh26jy]

I’ve started experimenting with basic gpu passthrough, windows VM and drivers. I did reboot of the hypervisor at one point and noticed that SR-IOV capabilities appeared on the GPU.

It may be worth adding to the tutorial that out of the box GPU may come with older internal firmware and without SR-IOV support. Windows drivers are capable of upgrading that firmware.

[IroesStrongarm]

@wendell

oops, I created a systemd service for this. will add it to the howto

I had a feeling that was the case. Figured that’s what I would do eventually but wasn’t sure if that was just my lack of understanding and making a cludgy workaround, or the actual proper way to do things.

Thanks for the confirmation on that.

[IroesStrongarm]

@IroesStrongarm

Tried getting this to work in an Ubuntu server VM. Didn’t work right but guessing that’s because I was on kernel 6.8 instead of 6.17. Will have to play around with it more.

Just setup an Ubuntu machine with kernel 6.17 and at least in lspci -vvv it appears all functionality is there now. Will likely install Jellyfin later just to test it’ll transcode but do believe it should work…

[ynfh26jy]

Basic gpu passthrough without sr-iov works flawlessly for me. I can run gpu benchmarks and play games.

But SR-IOV is complete mess. I’ve installed gfx_win_101.6979.exe drivers and I cannot finish superposition benchmark in directx mode. These drivers show date 19.08.2025 in task manager.
These are the newest drivers I see on the intel website.

I’ve tried to use intel graphics software to update the drivers. It had some issues, it was showing that newer drivers were available but stopped. (errors or did someone from intel pulled them? dunno)

I have been testing with apollo sunshine fork because it has decent virtual display. General, non-gamming, VDI experience is amazing. Wendell said in video that he couldn’t get encoders to work in sr-iov. I have been running 2 VMs with apollo to moonlight streaming and something was happening on “video decode” graph in task manager on both of them.

Apollo log also shows that it’s using hevc_qsv encoder in both running VMs.

Overall GPU is very impressive. SR-IOV can be easily reconfigured online without hypervisor shutdown. I just hope they will fix the drivers because I couldn’t launch any 3d game I’ve tried. (all of these games worked in non sr-iov passthrough)

[wendell]

try the older driver from the video. completes superposition fine.

[ynfh26jy]

In the video you just refer to it as older driver. (maybe I’m missing something) Can you specify which exact version works in your setup?

Also, in the tutorial in this thread you link to Intel Arc Pro drivers but screenshot shows Intel Arc non-Pro download page. Which drivers are you using?
Is there a functional difference between these drivers?

[wendell]

its in the op, with link even

[ynfh26jy]

Ok, I had correct version all along.
I assumed you were talking about older version of the pro driver. Nevermind.

gfx_win.101.6979.exe aka 32.0.101.6979
I’ve reinstalled it on VMs and did he firmware install just to be sure.

Superposition is unstable when I have 1 SR-IOV VM running. It failed 2 out of my last 5 attempts in this scenario.

Superposition crashes shortly after I start something graphically intensive in the other VM. (both VMs have the same drivers)
I had doom 2016 running ok-ish in one VM. But trying to start 2nd doom in another VM crashes one or both games.

I’ll revisit SR-IOV once intel releases some updates.

[wendell]

try adding iommu=pt to your kernel line?

[ynfh26jy]

I had intel_iommu=on and added iommu=pt but it didn’t change anything.

Superposition opengl seems to be more reliable than directx version. Vulkan doom 2016 at least launches. Directx seems to be more broken than alternatives.

Just for fun, attempting to launch doom in second VM. (at 800x600 resolution in game)

[long.barn4823]

I have Asrock B60

I have booted to windows and install driver …6979

sriov enabled in bios, iommu on in grub … no sriov capability on gpu, yet. Any word on if we think this should be working like B50?

03:00.0 VGA compatible controller: Intel Corporation Battlemage G21 [Intel Graphics] (prog-if 00 [VGA controller])
	Subsystem: ASRock Incorporation Device 6023
	Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-
	Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
	Latency: 0, Cache Line Size: 64 bytes
	Interrupt: pin ? routed to IRQ 232
	IOMMU group: 20
	Region 0: Memory at 82000000 (64-bit, non-prefetchable) [size=16M]
	Region 2: Memory at 4000000000 (64-bit, prefetchable) [size=32G]
	Expansion ROM at 83000000 [disabled] [size=2M]
	Capabilities: [40] Vendor Specific Information: Intel Capabilities v1
		CapA: Peg60Dis- Peg12Dis- Peg11Dis- Peg10Dis- PeLWUDis- DmiWidth=x4
		      EccDis- ForceEccEn- VTdDis- DmiG2Dis- PegG2Dis- DDRMaxSize=Unlimited
		      1NDis- CDDis- DDPCDis- X2APICEn- PDCDis- IGDis- CDID=0 CRID=0
		      DDROCCAP- OCEn- DDRWrtVrefEn+ DDR3LEn+
		CapB: ImguDis- OCbySSKUCap- OCbySSKUEn- SMTCap- CacheSzCap 0x0
		      SoftBinCap- DDR3MaxFreqWithRef100=Disabled PegG3Dis-
		      PkgTyp- AddGfxEn- AddGfxCap- PegX16Dis- DmiG3Dis- GmmDis-
		      DDR3MaxFreq=2932MHz LPDDR3En-
	Capabilities: [70] Express (v2) Endpoint, IntMsgNum 0
		DevCap:	MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 unlimited
			ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset+ SlotPowerLimit 0W TEE-IO-
		DevCtl:	CorrErr- NonFatalErr- FatalErr- UnsupReq-
			RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ FLReset-
			MaxPayload 256 bytes, MaxReadReq 512 bytes
		DevSta:	CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
		LnkCap:	Port #0, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <64ns, L1 <1us
			ClockPM- Surprise- LLActRep- BwNot- ASPMOptComp+
		LnkCtl:	ASPM L1 Enabled; RCB 64 bytes, LnkDisable- CommClk-
			ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- FltModeDis-
		LnkSta:	Speed 2.5GT/s, Width x1
			TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-
		DevCap2: Completion Timeout: Range B, TimeoutDis+ NROPrPrP- LTR+
			 10BitTagComp+ 10BitTagReq+ OBFF Not Supported, ExtFmt+ EETLPPrefix-
			 EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
			 FRS- TPHComp- ExtTPHComp-
			 AtomicOpsCap: 32bit- 64bit- 128bitCAS-
		DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis-
			 AtomicOpsCtl: ReqEn-
			 IDOReq- IDOCompl- LTR+ EmergencyPowerReductionReq-
			 10BitTagReq- OBFF Disabled, EETLPPrefixBlk-
		LnkCap2: Supported Link Speeds: 2.5GT/s, Crosslink- Retimer- 2Retimers- DRS-
		LnkCtl2: Target Link Speed: 2.5GT/s, EnterCompliance- SpeedDis-
			 Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
			 Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot
		LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1-
			 EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest-
			 Retimer- 2Retimers- CrosslinkRes: unsupported, FltMode-
	Capabilities: [ac] MSI: Enable+ Count=1/1 Maskable+ 64bit+
		Address: 00000000fee010b8  Data: 0000
		Masking: 00000000  Pending: 00000000
	Capabilities: [d0] Power Management version 3
		Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-)
		Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
	Capabilities: [100 v1] Alternative Routing-ID Interpretation (ARI)
		ARICap:	MFVC- ACS-, Next Function: 0
		ARICtl:	MFVC- ACS-, Function Group: 0
	Capabilities: [110 v1] Null
	Capabilities: [200 v1] Address Translation Service (ATS)
		ATSCap:	Invalidate Queue Depth: 00
		ATSCtl:	Enable-, Smallest Translation Unit: 00
	Capabilities: [420 v1] Physical Resizable BAR
		BAR 2: current size: 32GB, supported: 256MB 512MB 1GB 2GB 4GB 8GB 16GB 32GB
	Capabilities: [400 v1] Latency Tolerance Reporting
		Max snoop latency: 15728640ns
		Max no snoop latency: 15728640ns
	Kernel driver in use: xe
	Kernel modules: xe

OS: Ubuntu 25.10 x86_64
Kernel: Linux 6.17.0-5-generic
CPU: Intel(R) Core(TM) i7-14700KF (28) @ 5.60 GHz
GPU 1: Intel Graphics @ 2.40 GHz [Discrete]
GPU 2: NVIDIA GeForce RTX 3060 [Discrete]

[ynfh26jy]

Slightly off topic hardware question.

Do you have any issues with fan on your B50?
Mine is usually really annoying and loud after enabling SR-IOV. lm_sensors gives me nonsensical RPM values on the host side.
It could be annoying for people trying to build quiet home servers/workstations.

No VMs attached, loud fan (probably max speed) and 0 RPM reported

xe-pci-0300
Adapter: PCI adapter
fan1:           0 RPM
pkg:          +39.0°C
vram:         +32.0°C
card:             N/A  (crit =   0.00 W, cap =  70.00 W)
card:          3.32 kJ
pkg:           1.79 kJ

Attaching VMs to virtual function doesn’t change that idle fan speed.

Lunching some load on the GPU seems to lower the fan speed?
Launching blander lowers fan speed for a while. Starting blander render with OneAPI acceleration seems to lower fan speed to inaudible (way below ambient level of my server room). I can feel heat coming out of the GPU but fan is barely pushing any air through it.

sensors output on Hypervisor during blender load

xe-pci-0300
Adapter: PCI adapter
fan1:        2900 RPM
pkg:          +67.0°C
vram:         +68.0°C
card:             N/A  (crit =   0.00 W, cap =  70.00 W)
card:         17.73 kJ
pkg:          11.63 kJ

Fan speed increases just after blender render finishes.

It seems that intel managed to Invert fan speed curve in SR-IOV mode.

Has anyone else observed this? Is this a driver/firmware issue or is my GPU faulty?

(I see similar behavior in non SR-IOV throughput with blender gpu render)

[ynfh26jy]

Did you reboot the host with the GPU after firmware upgrade?
I think on mine B50 SR-IOV appeared after I upgraded the firmware and made a full server reboot.

[wendell]

are there any clues on the output of dmesg when starting this process? what is the host platform and motherboard? you may have something that’s legit broken.

[Strawberry]

My B50 does this as well. And it is annoying but makes sense. If you have ASPM enabled and working, the host B50 PCIE link gets shut off when there is no load from any participating SR-IOV device. When the card sees this, it gets confused and the firmware takes over fan duty entirely, as it would if you put the card into a system that couldn’t initialize the PCIE link. You can’t monitor it from the hypervisor in this state, because the link is shut off and lm-sensors isn’t able to wake it up just to get the fan speed.

This feels like something that can be fixed with a firmware update. In the meantime, I suspect disabling power management will help (but I am not about to test that myself…) OR you can use my workaround which is to run something in one of the guests that will keep the card active but not doing anything. I have been using hwinfo64 in Windows, which polls the device periodically and keeps it active.

[ynfh26jy]

Dmesg looks ok I guess. I see no smoking gun

boot:

dmesg |grep -i -e xe -e iommu
[    0.000000] Command line: BOOT_IMAGE=/root/boot/vmlinuz-6.17.3-gentoo-dist root=UUID=bcebfd65-8733-4001-84eb-a49b68353f23 ro rootflags=subvol=root rootflags=compress=zstd console=ttyS0,115200 console=tty1 quiet iommu=pt intel_iommu=on
[    0.000000] NX (Execute Disable) protection: active
[    0.000555] MTRR map: 5 entries (3 fixed + 2 variable; max 23), built from 10 variable MTRRs
[    0.218946] Kernel command line: BOOT_IMAGE=/root/boot/vmlinuz-6.17.3-gentoo-dist root=UUID=bcebfd65-8733-4001-84eb-a49b68353f23 ro rootflags=subvol=root rootflags=compress=zstd console=ttyS0,115200 console=tty1 quiet iommu=pt intel_iommu=on
[    0.219001] DMAR: IOMMU enabled
[    0.486127] DMAR-IR: IOAPIC id 2 under DRHD base  0xfed91000 IOMMU 0
[    0.495289] ... fixed-purpose events:   4
[    0.009692] ... fixed-purpose events:   3
[    0.760545] pci 0000:00:1f.4: BAR 4 [io  0xefa0-0xefbf]
[    0.781985] iommu: Default domain type: Passthrough (set via kernel command line)
[    0.835116] pci 0000:00:00.0: Adding to iommu group 0
[    0.835130] pci 0000:00:01.0: Adding to iommu group 1
[    0.835140] pci 0000:00:06.0: Adding to iommu group 2
[    0.835149] pci 0000:00:0a.0: Adding to iommu group 3
[    0.835159] pci 0000:00:0e.0: Adding to iommu group 4
[    0.835175] pci 0000:00:14.0: Adding to iommu group 5
[    0.835183] pci 0000:00:14.2: Adding to iommu group 5
[    0.835203] pci 0000:00:15.0: Adding to iommu group 6
[    0.835212] pci 0000:00:15.1: Adding to iommu group 6
[    0.835220] pci 0000:00:15.2: Adding to iommu group 6
[    0.835232] pci 0000:00:16.0: Adding to iommu group 7
[    0.835240] pci 0000:00:17.0: Adding to iommu group 8
[    0.835257] pci 0000:00:1a.0: Adding to iommu group 9
[    0.835279] pci 0000:00:1b.0: Adding to iommu group 10
[    0.835289] pci 0000:00:1b.4: Adding to iommu group 11
[    0.835311] pci 0000:00:1c.0: Adding to iommu group 12
[    0.835321] pci 0000:00:1c.1: Adding to iommu group 13
[    0.835339] pci 0000:00:1c.2: Adding to iommu group 14
[    0.835354] pci 0000:00:1d.0: Adding to iommu group 15
[    0.835370] pci 0000:00:1d.4: Adding to iommu group 16
[    0.835395] pci 0000:00:1f.0: Adding to iommu group 17
[    0.835404] pci 0000:00:1f.3: Adding to iommu group 17
[    0.835414] pci 0000:00:1f.4: Adding to iommu group 17
[    0.835424] pci 0000:00:1f.5: Adding to iommu group 17
[    0.835435] pci 0000:01:00.0: Adding to iommu group 18
[    0.835447] pci 0000:02:01.0: Adding to iommu group 19
[    0.835458] pci 0000:02:02.0: Adding to iommu group 20
[    0.835656] pci 0000:03:00.0: Adding to iommu group 21
[    0.835667] pci 0000:04:00.0: Adding to iommu group 22
[    0.835677] pci 0000:05:00.0: Adding to iommu group 23
[    0.835694] pci 0000:06:00.0: Adding to iommu group 24
[    0.835731] pci 0000:08:00.0: Adding to iommu group 25
[    0.835759] pci 0000:08:00.1: Adding to iommu group 26
[    0.835769] pci 0000:0a:00.0: Adding to iommu group 27
[    0.835787] pci 0000:0b:00.0: Adding to iommu group 28
[    0.835798] pci 0000:0d:00.0: Adding to iommu group 29
[    1.084349] evm: security.SMACK64EXEC (disabled)
[   22.351788] RAPL PMU: API unit is 2^-32 Joules, 2 fixed counters, 655360 ms ovfl timer
[   22.616288] xe 0000:03:00.0: vgaarb: deactivate vga console
[   22.616488] xe 0000:03:00.0: [drm] Running in SR-IOV PF mode
[   22.616554] xe 0000:03:00.0: [drm] Found battlemage (device ID e212) discrete display version 14.01 stepping B0
[   22.618290] xe 0000:03:00.0: [drm] VISIBLE VRAM: 0x0000004000000000, 0x0000000400000000
[   22.618314] xe 0000:03:00.0: [drm] VRAM[0, 0]: Actual physical size 0x0000000400000000, usable size exclude stolen 0x00000003fb000000, CPU accessible size 0x00000003fb000000
[   22.618315] xe 0000:03:00.0: [drm] VRAM[0, 0]: DPA range: [0x0000000000000000-400000000], io range: [0x0000004000000000-43fb000000]
[   22.618317] xe 0000:03:00.0: [drm] Total VRAM: 0x0000004000000000, 0x0000000400000000
[   22.618319] xe 0000:03:00.0: [drm] Available VRAM: 0x0000004000000000, 0x00000003fb000000
[   22.649996] xe 0000:03:00.0: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=none:owns=none
[   22.651667] Loading firmware: xe/bmg_guc_70.bin
[   22.653487] xe 0000:03:00.0: [drm] GT0: Using GuC firmware from xe/bmg_guc_70.bin version 70.49.4
[   22.663595] xe 0000:03:00.0: [drm] Finished loading DMC firmware i915/bmg_dmc.bin (v2.6)
[   22.753188] xe 0000:03:00.0: [drm] GT0: ccs1 fused off
[   22.753191] xe 0000:03:00.0: [drm] GT0: ccs2 fused off
[   22.753192] xe 0000:03:00.0: [drm] GT0: ccs3 fused off
[   22.773001] Loading firmware: xe/bmg_guc_70.bin
[   22.773274] xe 0000:03:00.0: [drm] GT1: Using GuC firmware from xe/bmg_guc_70.bin version 70.49.4
[   22.774574] Loading firmware: xe/bmg_huc.bin
[   22.783196] xe 0000:03:00.0: [drm] GT1: Using HuC firmware from xe/bmg_huc.bin version 8.2.10
[   22.794083] xe 0000:03:00.0: [drm] GT1: vcs1 fused off
[   22.794084] xe 0000:03:00.0: [drm] GT1: vcs3 fused off
[   22.794084] xe 0000:03:00.0: [drm] GT1: vcs4 fused off
[   22.794085] xe 0000:03:00.0: [drm] GT1: vcs5 fused off
[   22.794085] xe 0000:03:00.0: [drm] GT1: vcs6 fused off
[   22.794086] xe 0000:03:00.0: [drm] GT1: vcs7 fused off
[   22.794087] xe 0000:03:00.0: [drm] GT1: vecs2 fused off
[   22.794087] xe 0000:03:00.0: [drm] GT1: vecs3 fused off
[   22.830815] xe 0000:03:00.0: [drm] Registered 4 planes with drm panic
[   22.830817] [drm] Initialized xe 1.1.0 for 0000:03:00.0 on minor 0
[   22.909706] xe 0000:03:00.0: [drm] Cannot find any crtc or sizes
[   22.931152] xe 0000:03:00.0: [drm] Using mailbox commands for power limits
[   22.931580] xe 0000:03:00.0: [drm] PL2 is supported on channel 0
[   22.948269] Creating 4 MTD partitions on "xe.nvm.768":
[   22.948288] 0x000000000000-0x000000001000 : "xe.nvm.768.DESCRIPTOR"
[   22.949358] 0x000000001000-0x00000054e000 : "xe.nvm.768.GSC"
[   22.950417] 0x00000054e000-0x00000074e000 : "xe.nvm.768.OptionROM"
[   22.952163] 0x00000074e000-0x00000075e000 : "xe.nvm.768.DAM"
[   23.069783] xe 0000:03:00.0: [drm] Cannot find any crtc or sizes
[   23.069796] snd_hda_intel 0000:04:00.0: bound 0000:03:00.0 (ops lmtt_ml_ops [xe])
[   23.149837] xe 0000:03:00.0: [drm] Cannot find any crtc or sizes

Enabling SR-IOV

[  199.129212] xe 0000:03:00.0: [drm] GT0: PF: VF1..VF2 provisioned with 2090348544 (1.95 GiB) GGTT
[  199.129952] xe 0000:03:00.0: [drm] GT0: PF: VF1..VF2 provisioned with 8468299776 (7.89 GiB) LMEM
[  199.130115] xe 0000:03:00.0: [drm] GT0: PF: VF1..VF2 provisioned with 32639 GuC context IDs
[  199.130245] xe 0000:03:00.0: [drm] GT0: PF: VF1..VF2 provisioned with 128 GuC doorbell IDs
[  199.130479] xe 0000:03:00.0: [drm] GT1: PF: VF1..VF2 provisioned with 32639 GuC context IDs
[  199.130604] xe 0000:03:00.0: [drm] GT1: PF: VF1..VF2 provisioned with 128 GuC doorbell IDs
[  199.232681] pci 0000:03:00.1: [8086:e212] type 00 class 0x030000 PCIe Endpoint
[  199.233181] pci 0000:03:00.1: Adding to iommu group 30
[  199.233203] pci 0000:03:00.1: vgaarb: no bridge control possible
[  199.233207] pci 0000:03:00.1: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[  199.233213] xe 0000:03:00.0: vgaarb: VGA decodes changed: olddecodes=none,decodes=none:owns=io
[  199.233457] xe 0000:03:00.1: enabling device (0000 -> 0002)
[  199.234809] xe 0000:03:00.1: [drm] Running in SR-IOV VF mode
[  199.236592] xe 0000:03:00.1: [drm] VF: migration not supported by this module version
[  199.239088] xe 0000:03:00.1: [drm] VISIBLE VRAM: 0x0000004400000000, 0x0000000200000000
[  199.239119] xe 0000:03:00.1: [drm] VRAM[0, 0]: Actual physical size 0x00000001f8c00000, usable size exclude stolen 0x00000001f8c00000, CPU accessible size 0x00000001f8c00000
[  199.239131] xe 0000:03:00.1: [drm] VRAM[0, 0]: DPA range: [0x0000000000000000-1f8c00000], io range: [0x0000004400000000-45f8c00000]
[  199.239143] xe 0000:03:00.1: [drm] Total VRAM: 0x0000004400000000, 0x00000001f8c00000
[  199.239150] xe 0000:03:00.1: [drm] Available VRAM: 0x0000004400000000, 0x00000001f8c00000
[  199.293150] xe 0000:03:00.1: [drm] GT0: ccs1 fused off
[  199.293153] xe 0000:03:00.1: [drm] GT0: ccs2 fused off
[  199.293154] xe 0000:03:00.1: [drm] GT0: ccs3 fused off
[  199.303425] xe 0000:03:00.1: [drm] GT1: vcs1 fused off
[  199.303432] xe 0000:03:00.1: [drm] GT1: vcs3 fused off
[  199.303435] xe 0000:03:00.1: [drm] GT1: vcs4 fused off
[  199.303438] xe 0000:03:00.1: [drm] GT1: vcs5 fused off
[  199.303441] xe 0000:03:00.1: [drm] GT1: vcs6 fused off
[  199.303444] xe 0000:03:00.1: [drm] GT1: vcs7 fused off
[  199.303448] xe 0000:03:00.1: [drm] GT1: vecs2 fused off
[  199.303451] xe 0000:03:00.1: [drm] GT1: vecs3 fused off
[  199.310435] [drm] Initialized xe 1.1.0 for 0000:03:00.1 on minor 1
[  199.310541] pci 0000:03:00.2: [8086:e212] type 00 class 0x030000 PCIe Endpoint
[  199.310649] pci 0000:03:00.2: Adding to iommu group 31
[  199.310655] pci 0000:03:00.2: vgaarb: no bridge control possible
[  199.310655] pci 0000:03:00.2: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none
[  199.310657] xe 0000:03:00.0: vgaarb: VGA decodes changed: olddecodes=none,decodes=none:owns=io
[  199.310686] xe 0000:03:00.2: enabling device (0000 -> 0002)
[  199.311407] xe 0000:03:00.2: [drm] Running in SR-IOV VF mode
[  199.311854] xe 0000:03:00.2: [drm] VF: migration not supported by this module version
[  199.313024] xe 0000:03:00.2: [drm] VISIBLE VRAM: 0x0000004600000000, 0x0000000200000000
[  199.313027] xe 0000:03:00.2: [drm] VRAM[0, 0]: Actual physical size 0x00000001f8c00000, usable size exclude stolen 0x00000001f8c00000, CPU accessible size 0x00000001f8c00000
[  199.313028] xe 0000:03:00.2: [drm] VRAM[0, 0]: DPA range: [0x0000000000000000-1f8c00000], io range: [0x0000004600000000-47f8c00000]
[  199.313030] xe 0000:03:00.2: [drm] Total VRAM: 0x0000004600000000, 0x00000001f8c00000
[  199.313031] xe 0000:03:00.2: [drm] Available VRAM: 0x0000004600000000, 0x00000001f8c00000
[  199.360690] xe 0000:03:00.2: [drm] GT0: ccs1 fused off
[  199.360692] xe 0000:03:00.2: [drm] GT0: ccs2 fused off
[  199.360693] xe 0000:03:00.2: [drm] GT0: ccs3 fused off
[  199.365371] xe 0000:03:00.2: [drm] GT1: vcs1 fused off
[  199.365373] xe 0000:03:00.2: [drm] GT1: vcs3 fused off
[  199.365373] xe 0000:03:00.2: [drm] GT1: vcs4 fused off
[  199.365374] xe 0000:03:00.2: [drm] GT1: vcs5 fused off
[  199.365374] xe 0000:03:00.2: [drm] GT1: vcs6 fused off
[  199.365375] xe 0000:03:00.2: [drm] GT1: vcs7 fused off
[  199.365375] xe 0000:03:00.2: [drm] GT1: vecs2 fused off
[  199.365376] xe 0000:03:00.2: [drm] GT1: vecs3 fused off
[  199.369348] [drm] Initialized xe 1.1.0 for 0000:03:00.2 on minor 2
[  199.369463] xe 0000:03:00.0: [drm] PF: Enabled 2 of 12 VFs

Launching VM with gpu VF (split locks appear only during boot and are probably harmless)

[  275.004488] VFIO - User Level meta-driver version: 0.3
[  275.427739] vfio-pci 0000:03:00.1: vgaarb: VGA decodes changed: olddecodes=io+mem,decodes=io+mem:owns=none
[  275.476285] tun: Universal TUN/TAP device driver, 1.6
[  275.477430] brvl5: port 2(vnet0) entered blocking state
[  275.477439] brvl5: port 2(vnet0) entered disabled state
[  275.477451] vnet0: entered allmulticast mode
[  275.477551] vnet0: entered promiscuous mode
[  275.477729] brvl5: port 2(vnet0) entered blocking state
[  275.477733] brvl5: port 2(vnet0) entered forwarding state
[  275.534685] /proc/cgroups lists only v1 controllers, use cgroup.controllers of root cgroup for v2 info
[  277.008666] vfio-pci 0000:03:00.1: enabling device (0000 -> 0002)
[  277.008674] vfio-pci 0000:03:00.1: resetting
[  277.009012] xe 0000:03:00.0: [drm] GT0: PF: VF1 FLR
[  277.009051] xe 0000:03:00.0: [drm] GT1: PF: VF1 FLR
[  277.112155] vfio-pci 0000:03:00.1: reset done
[  277.118904] vfio-pci 0000:03:00.1: resetting
[  277.119019] xe 0000:03:00.0: [drm] GT0: PF: VF1 FLR
[  277.119113] xe 0000:03:00.0: [drm] GT1: PF: VF1 FLR
[  277.225621] vfio-pci 0000:03:00.1: reset done
[  285.058483] x86/split lock detection: #AC: CPU 0/KVM/12879 took a split_lock trap at address: 0x758c86cf
[  285.068735] x86/split lock detection: #AC: CPU 1/KVM/12880 took a split_lock trap at address: 0x758c86cf
[  285.140418] x86/split lock detection: #AC: CPU 3/KVM/12882 took a split_lock trap at address: 0x758c86cf
[  285.525710] x86/split lock detection: #AC: CPU 2/KVM/12881 took a split_lock trap at address: 0x758c6379

and that’s it for dmesg.
This time I’ve launched single VM with 8GB of vram
I’ve managed to launch System shock remake, played for few minutes. Lag happened in game and the game crashed (windows and other programs were fine). There were no dmesg messages or errors.

Machine:
CPU: intel i5 13500
Mobo: asus prime z790-p ddr5
RAM: CMK192GX5M4B5200C38 (4x48GB @ 3600MT/s) 192GB
Corsair rm550x 550W PSU
OS: gentoo
Kernel: 6.17.3-gentoo-dist
Qemu: 9.2.3-r3 ( I’ve also tested 10.0.5 )
Libvirt: 11.1.0

I have been using it as a hypervisor for about a year and it has been rock solid ( on 3600 MT/s ram settings). Right now it’s just a test system.

Maybe next month I’ll have some spare hardware and time to test B50 on actual proxmox. But I’m skeptical since basic building blocks are mostly the same.

I’ve tested also older heaven benchmark you used.
It’s more stable, I had only one crash. 2nd VM has 1/3 of the FPS without using window moving trick you showed.

I’ll run heaven or valley bencmark overnight on non-SR-IOV B50 VM to test how stable it is in more normal scenario.

[strh]

Thanks to all for a really helpful thread/guide.

Not sure if there’s another place to record early “draft” successes, but here’s what I have working.

Motherboard: X11SPH-nCTPF
CPU: Xeon Gold 6226R
MB Bios: 4.7 (PATCHED)
GPU: b50: Install driver gfx_win_101.6979 (appears to upload new firmware to card) followed by gfx_win_101.8136 after a reboot.
GPU: b50: Enabled PCI Passthrough in ProxMox to get GPU working in Guest. Without patching MB Bios this did NOT work.
GPU: b50: Enabled SR-IOV Passthrough to test passing through 1 of 4 virtual GPUs to Guest.
HOST OS: Proxmox (Linux 6.17.1-1-pve)
GUEST OS: Windows 10

As mentioned, I did have to patch the MB bios to enable Re-Bar support and thus allow PCI passthrough of the b50 to guests, without triggering the Code 43 error.

I’m continuing to test.

Cheers!