Has anyone configured Mellanox ConnectX4 BO-ISCSI via UEFI boot?

Hi All,

I’m looking for some expert help here. I am trying to configure ISCSI boot from a Precision 5810 with a Mellanox Connect X4 LX 10/25GBE dual nic.

I want to network boot my ESXI installation to free up some PCIE lanes on the workstation. I have read through all of the Mellanox documentation and have gotten nowhere. I see that you can use the legacy Flexboot to Boot over ISCSI, and via the documentation you should be able to use UEFI and SecureBoot. However, I can not seem to find a way to configure this. All that I get when I select the nic in the UEFI is the Boot from UEFI PXE.

Does anyone have any tips or suggestions? I know this is a very odd configuration and it likely makes zero sense to most people. Thanks

Not an answer, but I’m curious, why not just use PXE? … why bother with iscsi for esxi itself?

I don’t believe you can setup a persistent volume when booting from PXE. You are essentially just hosting iso files from your TFTP server.

Also ISCSI is a more efficient protocol for this type of activity.

from this docs it looks like not all mellanox firmwares have flexboot support, make sure yours does?

To configuring your client:

  1. Verify the Mellanox adapter card is updated with the correct Mellanox FlexBoot version…
    For boot over Ethernet, when using adapter cards with older firmware version than 2.30.8000,
    you need to burn the adapter card with Ethernet FlexBoot, otherwise use the VPI FlexBoot.
  2. Verify the Mellanox adapter card is burned with the correct firmware version.
  3. Set the “Mellanox Adapter Card” as the first boot device in the BIOS settings boot order.

Pxe is the protocol that allows you to run a bootloader and an initial kernel from a network resource, and it requires a tftp server running, flexboot is mellanox version of ipxe that boots from the bios, so no pxe/DHCP/ network config is required, it allows you to go straight to booting a kernel from a San volume …

So

… flexboot lets you skip one link in the bootloader chain compared to e.g serving up ipxe over tftp?

… and ibft is the only way esxi can run diskless?

… and esxi the goes looking for your iqn and other things in whatever ibft data gets passed to it, after being populated by ipxe.

… and all potentially useful engineering documentation on what mellanox/nvidia does and what esxi expects has not been published and was instead replaced with marketing buzzwordy lingo bingo?

  1. Yes
  2. Yes, seems so.
  3. Correct. The connectx card really should be handling most of that and passing a lun directly to the UEFI as a “local” storage device.
  4. Correctamundo! Basically all of it covers the old Legacy option ROMs. However, you’d be crazy to go that route nowadays. UEFI is the way to go for all installs if you can do it.

It does have flexboot support in this firmware. I am able to access it via legacy mode, but not UEFI. I also used the MLXUP utility to bring the card to the latest firmware version.

It is a MCX4121A-ACAT running FW: 14_13_1010
UEFI: 14.25.17
FlexBoot: 3.6.502

The doc I linked, at page 164, shows where in the bios you can set the iscsi parameters … the question is whether the precision bios allows you to do that…

Hmmm… That is what I feared… I already found that the Precision 5810 does not fully support SR-IOV.

I will check it out and see. Thanks!

Well, if the bios config is not there, you can still boot an ipxe USB stick and get the iscsi thing going from there…