A Neverending Story: PCIe 3.0/4.0/5.0 Bifurcation, Adapters, Switches, HBAs, Cables, NVMe Backplanes, Risers & Extensions - The Good, the Bad & the Ugly

Wendell used these settings (refer to the YT video mentioned from me above)

That looks like all off (1) which is a bummer because I don’t think my system works with that.

Are you sure he used these settings or is this just b-roll?

Edit: @wendell hey… mind if I copy your homework here? Did you get it working with them all set to off (1)?

Yes

I have the OCuLink one, but I concur.

This is solid for PCIe 4.0 over 50 centimeters of OCuLink cabling.

5 Likes

What could suck is that maybe your adapter is just DOA, my first shipment from MicroSATACables had defective ones as well.

Do you have a second one and a different cable?

Tried it with a different drive. It worked fine. I think I broke the drive itself earlier : /

Anyway thanks for the help.

Cyborgmatt-Question-1

I recently purchased a U2 (SFF-8643) to SFF-8087 cable to connect to my server chassis` backplane using the U2 port on the motherboard. When I change the U2 mode in the BIOS from PCIe to SATA all four HDDs showed up however the NVMe drive I have in M2_2 completely disappeared from the BIOS.

I checked the motherboard manual and all I can see is the following:

The M.2_2 socket shares PCIe lanes with PCIeX1_1; when PCIeX1_1 is occupied, M.2_2 only can only run at PCIE 4.0 x 1.

I don’t have anything in the PCIeX1_1 slot so I’m confused, there are also no references to the U2 port disabling the second M2 slot in the manual.

Motherboard: ASUS Pro WS X570-ACE
CPU: Ryzen 5900x
PCIe Slot #1: 1080Ti
PCIe Slot #3: Dell H310 Lsi 9240-8i
M2_1: 1TB Corsair MP600 PRO
M2_2: 1TB Corsair MP600 PRO

Is this intended behaviour? I can’t seem to find any one else with this issue so I’m looking for a sanity check.

Welcome Cybordmatt!

No, I cannot confirm this behavior with both same mobo and CPU.

I have both PCIe slot PCIeX1_1 and M2_2 occupied and have switched multiple times between SATA and NVMe mode on the U2 slot without loosing access to M2_2 (my boot drive - I would notice).

Where in the BIOS do you expect to see the M2_2 drive? Have you booted into an OS (which one) and looked for it there?

1 Like

No, as @jode mentioned definitely not normal behavior.

Have you already updated to the latest BIOS (4501 from late May 2023)?

I’m using multiple ASUS Pro WS X570-ACE with every PCIe slot/port occupied and would have noticed that, too.

Other settings that might help recognizing all PCIe devices:

1 Like

Hey Jord and Normie, thanks for the fast reply. The BIOS settings you posted look identical to mine. I’m on the latest BIOS update 4501, I even rolled back to older versions to see if that would help (it didn’t).

Here are some screenshots that help convey what I mean about the drive “disappearing”.

Before plugging in the Mini SAS HD U.2 cable:


upload://iFhUOTM1AfuqACylKHeZHh0A2uO.png (Screenshot showing both M2 drives in the SSD Secure Erase list)
upload://uCP8bgWuQESu3CCTaPEVxpPuMo.png (Screenshot showing both drives listed on the Adv\NVMe Controller & Drive Info page)

After plugging in the Mini SAS HD U.2 cable:


upload://zaxplFQpR7xV3nZbOpAhf1t6fzn.png (Screenshot showing 2nd M2 drive missing in the SSD Secure Erase list)
upload://4hQgaGWtCtNsT2MaowUl8a1b0dr.png (Screenshot showing 2nd M2 drive missing on the Adv\NVMe Controller & Drive Info page)

Confirming that Unraid can no longer see the second NVMe drive:

upload://eY0JrSWaNqijANCt43vV8RgxD10.png (Screenshot showing Unraid unable to detect the 2nd M2 drive)

Honestly, I’m stumped.

Edit: Unfortunately I can’t post direct links or include more than two images because my account is new.

At the top of my head I don’t see anything obviously wrong that might trigger such a behavior.

I’ll try to recreate a similar setup on one of my boards (using the U.2 port for PCIe, not for SATA, have to get an SFF-8643-to-4xSATA cable first).

In the meantime could you try 3 things?

  1. Switch the position of the two M.2 SSDs to check if the behavior remains unchanged

  2. Remove the LSI HBA PCIe from the second CPU PCIe slot to check if the HBA is causing buggy behavior. Since this is CPU PCIe and M2_2 is chipset PCIe these things shouldn’t be related in any way but it’s better to check than to maybe miss something.
    (3ware became LSI, LSI became AVAGO, AVAGO became Broadcom and Broadcom has become shyte.)

  3. Check your motherboard CMOS battery, this motherboard model has been out for almost four years and depending on the manufacturing date the battery might be done for. I’ve also seen motherboards less than a year old (since manufacturing) with faulty batteries.

1 Like
  1. Swapped the two M2 drives around.

  2. Completely disconnect the HBA card (removed it from the slot and disconnected the cables).

  3. Installed a brand new CMOS battery.

Sadly still the same issue.

Just for another sanity check, I grabbed a spare SFF-8087 to 4x SATA reverse cable I had. Hooked that up to the four SATA ports on the motherboard and into the 8087 backplane. Both M2 drives show up and the four HDDs in the chassis from that row.

If that works then I really can’t understand why the U.2 port wouldn’t work if the motherboard’s block diagram I found is accurate.

I understand that these cables are directional. Can it be that you have a breakout cable and a reverse-breakout cable?

If you’re referring to my U2 Mini SAS cable, the cable itself is fine, the drives work and the motherboard sees them. The issue stems from the secondary M2 slot on the motherboard being randomly disabled when using it.

Nevermind. I read your post incorrectly.

Sorry that everything so far seems to have been in vain. Will report back when a suitable SFF-8643-to-4xSATA cable is at my disposal.

Just from my gut feeling I think you might have found a new bug with the motherboard that possibly needs to be addressed with a BIOS update by ASUS.

No problem, really appreciate all the help. Do you think it’s worth going through ASUS support with this issue?

Also, I don’t mind sending you a cable if you need one. The least I could do for your help.

Thanks but that’s not necessary, just have to wait a bit until I can power down a system that has quite a few of these cables inside :wink:

You should always report issues to manufacturers, mistakes happen and by the reaction from them you can actually judge the character of a company. That’s how Broadcom has landed on my personal black list.

PCIe gen5 retimer adapters have hit the retail channels:

Retimers should be way more forgiving than the pcie 4 redrivers because they don’t amplify the undesirable portions of the signal.

5 Likes