While I know that the HBAs formerly known as LSI/AVAGO now Broadcom have been the industry standard for over a decade I seem to bump into weird stuff with them. I have been using two units of the HBA 9400-8i8e for homelab use for two years and of course, just when I was finalizing a build I’m now experiencing new issues:
The HBA and its driver supports standby S3 mode of the host system; had been working fine for almost four years.
New issue: After a power cycle the system is running fine and can enter S3 mode (and wake up again), initiate a reboot or a shutdown via the operating system (tested Windows 10 x64 21H2 and Linux).
BUT: After the first use of the S3 mode and successfully waking up, the system crashes with a BSOD when trying to enter S3 again or do a reboot or a shutdown, the system is in the process of rebooting/shutting down for a few minutes, then also crashes with a BSOD.
Yes, the categories on the screenshot show the correct selection of parameters to get to the HBA 9400 8i8e downloads, at least it had been in the past.
When selecting “All” in the last two drop-down selections you can still see the various files for the 9400 models (P22 are the latest releases, tried it out but didn’t change a thing)
I then tried the second HBA 9400-8i8e I have and it’s behaving exactly the same. If I remove the HBA the systems work completely fine again.
If I understood correctly this was working but now is having a problem. Or 2 years later you’re just starting to try/test S3 mode?
If everything was working and you’ve tried two different cards I’d tend to not think it’s the card specifically but something else in the environment like updates to drivers or the OS itself.
You could try booting from a thumb drive or USB3 SSD starting with a build you know was working previously without doing updates. If that works try updating the OS on the boot disk to see if it breaks. Same with any drivers. If the OS and all drivers are updated and the card works correctly you narrowed down a lot and can focus on config changes or other software.
But for something like this if previously working would be trying to think if any bios changes were made recently. I’d even consider printing out each page in the bios for reference and resetting it to default and trying again.
But what remains strange is that Broadcom designated not only the HBA 9400 models from 2017 to be legacy but also 2020’s HBA 9500 line that features PCIe Gen4 and more importantly doesn’t even have a successor yet
They seem to have announced the 96xx series.
I’m still waiting for my HBA 9500-16i . Expected delivery date is Mid July.
I wonder if I should just cancel the order, since currently I’m fine, as I found some used older SAS/SATA HBA for HDD, and got a second HyperX 4x M.2 card for SSDs.
You seem to have quite a few issues with the newer Broadcom HBAs.
Is Microchip/Adaptec any better?
The only non-Broadcom products I’m currently using are Intel RES3TV360 SAS3 expanders that use a PM8044 SXP chipset and are functioning fine with older Broadcom 9300 HW RAID controllers.
So I cannot say anything regarding other HBAs but it’s true ever since I got some HBA 9400 I seem to be cursed as soon as the name “Broadcom” is involved.
I’d hope that they maybe roll out firmware fixes since on paper these HBAs are pretty fine.
A thing that keeps me sane is that other users can recreate my experiences, the most recent example is broken firmware on an exotic Broadcom P411W-32P PCIe Switch NVMe HBA with @Illumous confirming the behavior with the latest firmware.
Sometimes I think that a higher power is trying to prevent me from moving from HW RAID to ZFS
I almost ordered the P411W-32P, but the ordered the 9500-16i, because going all NVMe would be very expensive. I’m currently running a single node ceph setup. The plan is once I get the 9500-16i card, and I could get the 16x PCIe to 2x 8x PCIe bifurcation cable to work, to move the current old HBA to my old ITX motherboard PC (together with mellanox 25GbE card) and use it as a second ceph node. So I’m reading your bifurcation thread as well
I wish you good luck and I’d be happy to read a short user experience “even if” your HBA 9500 works without any issues.
I got bitten by the NVMe bug for a local workstation and the desire to use multiple NVMe SSDs with the ability to eject and hot-plug them in a backplane without having to reboot Windows.
If you don’t want to lose performance (Thunderbolt- or USB-to-NVMe adapters), active PCIe Switch Adapters like the Delock 90504 for PCIe Gen3 or the P411W-32P for PCIe Gen4 are the only options to accomplish that as far as I know.
