Curious if anyone has put a Mellanox network card on the WRX90e-Sage motherboard yet. I’m having difficulty getting mine to work, part # MCX4121A-ACAT.
It seems to really hang during power on/reboot - as in it won’t post. Seems to get stuck on a couple different codes. Sometimes it’ll move on, other times I’ve waited over 5 minutes with no progression.
This is my first workstation board, and it seems to take an eternity to post even with the card removed. I have no scientific data to back it up but it feels longer whenever power is restored or I had to hard shut it down by holding power button. I guess I’m a bit impatient already.
I cannot get the card to be recognized by proxmox.
Things that work: an intel SFP+ card and an intel i350-t4.
The Mellanox card is recognized in a very old workstation PC, although I cannot get it to work. This could be because it is only PCI2.0 but at least it shows up in Windows device manager. It complains about not enough resources.
I’ve tried a couple different slots in the asus motherboard - no change in symptoms.
Maybe the card has been flashed with a firmware that is using BIOS instead of UEFI to initialize.
You could try enable CSM in Bios and enable legacy/Bios mode for this card and see if that helps to get the card recognized
When turning CSM on (Mellanox card removed) the monitor won’t output anything at all (GTX1080, HDMI → DVI, probably a 1280x1024 monitor) during boot, post, or within bios. I can access bios via IPMI and the monitor is still in standby.
When proxmox is running the top half of screen is all black, the bottom half is random pixel noise/“snow”. Nothing resembles graphics or a terminal login for proxmox.
I changed a bunch of the CSM settings to [UEFI only] or [Ignore] but I get the same results.
CSM seems totally broken.
I’ll do some digging, not even sure if I can reflash the firmware on a NIC.
I put the Mellanox card into my desktop PC. It booted, fully recognized.
I ran mlxfwmanager in administrative command prompt. It showed there was an update. Doesn’t look like it had any UEFI firmware at all. The old firmware appeared to be circa 2017, new firmware Dec 2021.
I issued the command again but with -u for update.
Put the card back in the WRX90 - boot seems smooth and reliable now. Cannot see in proxmox still. This is progress though.
I installed Windows 10 LTSC 2021, Mellanox recognized without need for drivers. That proved everything should work.
Went back to proxmox… found that there were errors registering BAR. Really no idea what that means, but the solution based on search results was to add pci=realloc=off to the kernel parameters. I’m a linux newb, so I struggled a bit with that. I’ve only gotten it to work by modifying the GRUB launch which I believe is one-time thing. Once that happened I took the win and called it quits for the night.
I did not come across documentation that the bottom PCIe slots are different. I’ll have to seek that out and read up on it. GPU is currently in slot 5 (if counting 1 through 7). No reason other than I wanted it out of the way to make it easier to load the ram modules after it booted with a single stick.
