A Neverending Story: PCIe 3.0/4.0/5.0 Bifurcation, Adapters, Switches, HBAs, Cables, NVMe Backplanes, Risers & Extensions - The Good, the Bad & the Ugly

Ohh I see. I can see this one and is shipped to Chile.

That one would replace this one right? https://es.aliexpress.com/item/1005007205208267.html?spm=a2g0o.cart.0.0.4d0938da7cG12N&mp=1&pdp_npi=5%40dis!CLP!CLP%2045500!CLP%2045500!!CLP%2042366!!!%40210318a717611846536686867eeede!12000039797920298!ct!CL!2965647439!!2!0&gatewayAdapt=glo2esp

And the others would work right? I want to buy before tomorrow, as after that it seems shipments will take a lot longer here to Chile.

Edit: I went it, wish me luck haha…

1 Like


This is the cabling topology I would suggest you use for x8x8 with the shinreal mcio stuff.

When you do x16 I have had to criss-cross the mcio cables for whatever reason.

2 Likes

Amazing, many thanks! I wonder how I could do X4/X4/X4/X4, didn’t found some adapters.

I plan first to do X16 to X8X8, but maybe do the other one later.

@EricM-Question-1

Is there a way to connect a Kioxia CD6 Gen 4 to a Gen 4 M.2 slot?

Probably with some adapter searching.
You would need some M.2 to miniSAS-HD/MCIO/etc. card and then the other end likely requires some SATA-Power to fulfill power requirements of U.2

I have done x4x4x4x4 with those shinreal adapters, just using mcio 8i to 2x mcio 4i or 2x oculink cables that split it. Corresponding bios configuration is still required.

1 Like

Amazing, will give it a go. Many thanks man for all your help! I hope in some weeks to be able to test the X8/X8 first.

Might have a new idea to get progress on the Broadcom P411W-32P front: *

Unfortunately I would need the help of someone with both models, a P411W-32P and a HBA 9500-16i and both genuine Broadcom models to hopefully get a Broadcom support case to open.

Any such unlucky folks here?

*My goal:

Get a firmware with all fixes for the P411W-32P that also works with directly-attached SSDs (so no need for an active electronic UBM backplane as is currently the case) and working, signed drivers for Windows.

Catastrophic bugs in Windows might help force Broadcom’s hand here. With two or more customers complaining about the issues it might maybe work…

@thr3e today I got the parts from aliexpress and it works flawlessly! Tested both X8/X8 and X16 5.0, without issues or loss. Tested on 1x5090 at X16 5.0 and 2x5090 each at X8 5.0.

I’m impressed, many thanks for the link of the J0MB adapter.

1 Like

Just to be sure: PCIe AER Support is enabled and verified to be working for the used PCIe Lanes?

1 Like

At least from short testing yes. Had to disassembly since in next weeks I will use this adapter on another motherboard.

Tested with CUDA-Z and some LLMs benchmarks to see errors, didn’t saw one so far. Sorry for not having pics though :frowning:

Been looking at stuff, and it seems that the dell 235NK is some kind of pcie switch for only 30-40 bucks right now. Anyone know whats up with it, or what’s wrong with it?

I have a few of those cards, and I haven’t seen their details posted anywhere else so I’ll share what I know here. Dell 235NK / 0235NK uses a PLX PEX 9733, a 33-lane, 9-port Gen3 switch, and has a full-height bracket. There’s also another version of the card, part number CDC7W / 0CDC7W, that is exactly the same hardware except that it comes with a half-height bracket instead of a full-height one. According to lspci, the card divides each SlimSAS 8i port into two x4 ports, but I haven’t actually tried to connect anything to them yet so I can’t confirm that configuration for certain.

I am having some hard time understanding this Dell 235NK - it is x16 and has only 2x SlimSAS ports, each capable of 2 drives. What exactly does the PLX switch do in this case ?

At least that’s what I see from the picture - only two SFF-8654 8i ports with each port usually being able to do 2x 4i devices. At least I am not aware of SFF-8654 8i to anything other than 2x 4i cables.

Am I missing something ?

1 Like

Can you please share the links with this information (or its in this thread)?

I tried searching this to see if any PCIe gen4 switches can be justified, but if they really waste 50W alone will be too much.

I’d expect the newer tech to be more efficient, even when they have to add more lanes, assuming they’re doing gen4 chips on a bit newer tech.

I am having some hard time understanding this Dell 235NK - it is x16 and has only 2x SlimSAS ports, each capable of 2 drives. What exactly does the PLX switch do in this case ?

Some systems can’t bifurcate their x16 PCIe slot to x4/x4/x4/x4, and can only bifurcate to x8/x8, x8/x4/x4, or x8/x4/x2/x2, or can’t bifurcate their x16 slot at all. There are also systems that have x16 slots that don’t have 16 lanes connected to them (e.g., a physical x16 connector with only one lane electrically connected to the slot) to have any bifurcation at all. The PCIe switch chip on the Dell 235NK enables connecting four x4 PCIe devices to these systems.

The PCIe switch also shortens the effective PCIe link to the drive, which helps preserve signal integrity. So unlike with a passive riser, where the PCIe lanes would be extended all the way from the CPU (or chipset) to the drive, the PCIe link from the CPU (or chipset) only goes to the PCIe switch, and then the PCIe switch has a new, separate link to the drive. A PCIe retimer would have a similar benefit, but retimers lack the ability to “split” a non-bifurcatable PCIe slot, and so are limited by the host’s ability to bifurcate the slot.

1 Like