Trouble enabling SRIOV on a NIC on VMware

Hardware Hub Networking
,
1

Hello, having a challenge enabling SR-IOV on a network card on VMware. The NIC is Intel® Ethernet Network Adapter XXV710-DA2 and Intel documentation indicates that it fully supports SR-IOV for virtualization. I’ve enabled SR-IOV on the BIOS to no avail. The server is a Dell R440. I’ve tried with both ESXI 8 and 7 and both indicate that the card doesn’t support SR-IOV. Any help would be appreciated.

esxi-sriov
esxi-sriov1268×358 23.1 KB

sriov-global
sriov-global1108×565 61.2 KB

sriov-nic
sriov-nic1102×383 26 KB

1 Like
2

Almost a year has passed since this was posted and the surprising lack of response seems to indicate that Level1 members don’t use and have little experience with SR-IOV, which is also surprising.

If members do can they suggest any cards that work flawlessly? If it turns out that the combination of NIC and motherboard is important please advise and also tell us your motherboard model. Thanks.

I have a Supermicro H12SSL-i with Broadcom onboard and SR-IOV doesn’t work even though enabled in BIOS and ESXi 8.0U2.

I’m about to buy a dual 1G-port NIC (10G SFP+ would be nice but no 10G switch) which needs to support SR-IOV. Can members suggest any cards for ESXi 8.0 and above?

3

Hey there. I have been using SRIOV on Esxi 7.0.3 for a few years now. I have not upgraded to 8.0 yet. However, in my experience, when the UI states that the PCI device is not capable of SRIOV like it shows in your screenshot, it means your driver is likely the issue.

I browse Broadcom’s support page for the latest driver for the NIC, install it via esxcli, and then the SRIOV function can be enabled.

I have personally setup SRIOV with Intel x520(10g), xl710(40g), xxv710(25g), and mellanox connectx-4 (both 25g and 100g variants) cards successfully on esxi 7.0.3.

Drivers used for Mellanox connectx-4
[root@vsphere1:~] esxcli software vib list | grep mlx
nmlx5-core 4.22.73.1006-1OEM.703.0.0.18644231 MEL VMwareCertified 2024-06-05
nmlx5-rdma 4.22.73.1006-1OEM.703.0.0.18644231 MEL VMwareCertified 2024-06-05

Driver used for Intel xl710 and xxv710 cards
[root@vsphere1:~] esxcli software vib list | grep i40
i40en 2.8.4.0-1OEM.700.1.0.15843807 INT VMwareCertified 2024-08-23

Driver used for intel x520 (82599)
[root@vsphere1:~] esxcli software vib list | grep ixg
ixgben 1.18.2.0-1OEM.700.1.0.15843807 INT VMwareCertified 2024-10-24

At least i confirmed these versions work with SRIOV. I made a point to do this process

  1. Install the new driver with ‘esxcli software vib install’ command.
  2. Reboot system. After bootup, you should see SRIOV no longer says “not capable” and rather says “disabled”
  3. Enable SRIOV on the NIC in the PCI devices page. Be sure to check the NIC driver for supported number of virtual functions. Setting the VFs too high will result in failure.
  4. Reboot system again.
  5. After this 2nd reboot, you should be able to assign VMs to the NIC virtual function devices.
  6. Edit network settings of a given VM. Add new network adapter and assign the port group. An SRIOV network adapter uses the vlan info in the port group settings to set the vlan tag or unagged. Change adapter type from VMXNET3(default) to SRIOV. Select the physical function PCI devices in the list. The VM will now use one of the virtual function devices.
  7. Reserve all VM memory.
  8. Disable VM CPU IOMMU support, disable all performance counters under the CPU section.
  9. Power up VM. You should have access to the SRIOV virtual function device in the VM.
4

Also, a few more things come to mind.

  1. BIOS settings need to be set correctly to allow for SRIOV to function. You have already found the global SRIOV setting in your dell BIOS. I check for these 3 things.

  • Enable virtualization technology toggle under System BIOS Setting → Processor Settings and enable Visualization Technology.

  • Enable the global SRIOV setting (you already found this one) under System BIOS Settings → Integrated Devices and enable SR-IOV Global Enable

  • In the BIOS, you can sometimes check specific features in the NIC settings. For Dell, you can enable SRIOV on the NIC itself by going to System Setup,. Click on the Device Settings and select the adapter. now set to SRI-OV enabled with 8 virtual functions.

  1. Another important thing to note is ESXi unlicensed does not support SRIOV. You have to have an enterprise plus license to enable SRIOV.