I have the Gigabyte WRX80 motherboard with a 3955WX, and I’m trying to add 4 GPU’s, a Highpoint RAID card (8x nvme), and the Gigabyte TR TB3 AIC. In Windows 10, I can get 2x GPU’s working with the TB3 card or 3x GPU’s by themselves alright, but any other combo seems to result in either a D4 error, or the POST codes just go through a never-ending cycle. Above 4G encoding is enabled and set to max (43 bit, 8TB), CSM is off, and secureboot on/off doesn’t seem to affect anything. Anything else I can try?
I should add that I’m on BIOS F2, which I can’t seem to update because I haven’t figured out IPMI and there’s no instructions on how to update BIOS without IPMI, despite having a Q-flash plus button on the motherboard. If anyone has pointers on that, I’d appreciate it. Gigabyte support said to follow instructions on some random external site and use the onboard USB port, but I can’t get it to work (yes, Gigabyte support doesn’t even have in house instructions on updating the BIOS on their own motherboards!).
IPMI should run off either of the GigE ports. I use the left one – that’s the first one. Sometimes I’ve needed to unplug and re-plug the cable in the GigE port to get a link.
By default, the IPMI will look to get its IP address off of a DHCP assignment. If you don’t have DHCP on the segment you’re connecting it to, you’ll need to go in the BIOS to manually assign the IP configuration:
Server Mgmt|BMC network configuration/Configuration Address source/Station IP address/Subnet mask/Router IP address
IPMI will be running on the https port of whatever IP address it gets.
It also takes a least 30 seconds for IPMI to come live after the power is first attached.
Starting on Page 42 here is an explanation of IPMI operation.
Page 51 has the Maintenance | BIOS updates walk throughs.
Do you have BNC VGA disabled [Jumper 1, page 16 in above manual] , since you have GPU(s) installed? If not, it might help lessen resource allocation demands.
However, it does look like BIOS F3 addresses some similar resource issues, so that might be a good bet too once you’re up on IPMI:
F3 9.27 MB 2021/04/22 4. Addressed MMIO out of resource issue when four graphic cards are installed
– WRX80-SU8-IPMI (rev. 1.0) Gallery | Motherboard - GIGABYTE Global
IPMI/BMC doesn’t want to seem to grab an IP address. The bmc led always blinks amber. Including powering off and removing the CMOS battery for a day. I really hope the mobo isn’t a dud, because I already replaced it due to a faulty pcie slot.
Can you try manually assigning an IP address in the BIOS, and then direct connect it to an Ethernet port (with compatible addressing) on your admin PC? (That should isolate any DHCP or connectivity variables, assuming your network cable is good).
I tried assigning an IP, but admit I might not have done that correctly. Pretty sure the cable and every other component is good, because they all work in relative isolation, including network access.
Disabling the BMC VGA is a good idea to avoid a conflict with the TB3 card.
I don’t want to be rude - but why are you buying a server board if you can’t assign an IP address in the BIOS, find the IP address assigned automagically by your DHCP server or use IPMI?
ASUS and Gigabyte must be going crazy with their WRX80 boards - I see so many posts about how people receive a bad board when it is really a lack of understanding of how these boards work and how to set them up.
That’s correct - it’s the ASpeed’s heartbeat and lets you know everything is good. The ASpeed chip is what allows you to talk to the board using IPMI over the LAN so you can configure things. Think of the ASpeed on the board as a mini computer that helps you monitor and configure things for you.
Make sense?
Get the IP address for the board, connect to it over your network using a browser. Once you get the IP address, connect using the HTTPS protocol - make sure you use the “S” version of “HTTPS,” as the connection needs to be secure.
Report back.
These boards are pretty spectacular - you just have to get used to a new way of thinking.
Figured out IPMI (my google chrome doesn’t like it, for some reason) and updated to F3 (updated firmware too). Tried disabling onboard VGA and still can only connect 2 GPU’s and TB3; OR 3x GPU’s and no TB3. Guess, I’ll roll the dice and hope F4 has some hidden benefit not described in the notes and also doesn’t mess with TB3 too much (apparently it might?).
Finally got it working, no thanks to Gigabyte support. After getting IPMI working, I updated BIOS to F3, which didn’t fix things by itself. Turning off fast boot disables CSM for some reason. Not sure why you can’t just disable CSM in the CSM menu. Anyway, now I have all my GPU’s working with TB3 on this thing. I’ll try enabling the AST VGA next.
Pushing back on these boards being spectacular. Despite having 128x PCIe 4.0 lanes, you still have to work to get devices to function because Gigabyte can’t allocate resources in a sensible manner? They couldn’t design this thing so the 1 PCIe lane the AST SoC uses (through the chipset, per the block diagram in the manual) doesn’t conflict with the the PCIe slots, which are direct connect to the CPU, btw. Also what’s with the onboard port placements? As in the USB port physically blocks most cards from using the last (#1) slot. I get that the edge slot slot can have issues with audio headers and whatnot, but this is a soldered on USB port that can’t be removed or just simply left unplugged. Also, the USB-c header blocks medium/log cards from seating properly in slot #3. What good is having all those PCIe slots if the BIOS won’t let you use them as well as having to decide between front headers and PCIe slots? I really want to like this board, but it’s been quite a headache to setup to it’s full potential. I would use the ASUS WRX80 board in a heartbeat if had TB3. That’s a pretty nicely designed motherboard, although definitely a big board.
I too had problems, and a bunch of learning curve frustrations. It seems like the AST VGA, which I was using rather than an GPU, conflicts with slot 2. I spent like a week troubleshooting Thunderbolt in slot 2, until I just tried it in slot 1 and it finally lit up as expected. But, yeah, looking at the schematic in the manual, everything should be separate/have it’s own path.
I put my Thunderbolt card in slot 1, and it’s a regular width card so it doesn’t intrude upon the space of the edge connectors. I do have some fans mounted
on the bottom of my case, so I did run into a problem using the USB 3 headers, but I bought these extenders, which allow me to move the connections down and into a hole to the bottom compartment of my case.
My case has a 90 degree USB 3.1 gen 2 motherboard header cable, so that works fine for me too, although again. I’m not using any PCIe cards that are more than single width cards.
I think when I finally get some GPUs for this build, I’ll shot for single slot water cooling GPU rigs, like these: https://www.ekfluidworks.com/workstations/
– but then, I’ll be more than happy just with a couple 3080 for the occasional hash cracking stuff I need to do
You’re probably dealing with a far more professional calculation rig.
Don’t you fret. I have the same issue on Asus WRX80E Sage . With 1 NVMe drive I can only plug 2 GPUs, as soon as I plug 3rd, D4 error - PCI resource allocation error. BIOS is at the newest version. Onboard VGA is disabled. Didn’t touch the bios settings yet. So I guess I have to dig in…
EDIT: So yeah, the culprit was the CSM enabled. After disabling it, no D4 error anymore…
I also have the Asus board and I’m having the D4 issue but only with CSM enabled in the bios. I have two GPUs, two of the hyper m.2 cards and a usb3 x1 card installed.
My issue is that with CSM enabled I’m able to get the GPUs to output on the physical display but only one card (with two it d4s). With CSM disabled the mb boots with all the cards, but I can’t get the GPUs to output anything on the HDMI or display ports when they are passed through to a Linux VM.
It would great to find a way to get it to boot with CSM enabled and get past the D4.
. With 1 NVMe drive I can only plug 2 GPUs, as soon as I plug 3rd, D4 error - PCI resource allocation error. BIOS is at the newest version. Onboard VGA is disabled. Didn’t touch the bios settings yet. So I guess I have to dig in…