I’ve been researching an unusual PCIe expansion project and thought you might be exactly the person to ask before I buy the hardware.
I found a seller with 20+ new/never-deployed Dell PowerEdge XE9680 donor chassis. They are incomplete servers, but each donor includes four Dell PCIe Switch Board assemblies and much of the original PCIe/power infrastructure.
I had the seller remove a heatsink from one of the PSBs and physically verify the switch:
Broadcom PEX89144B0-2
SS26-0B00-02
PCIe Gen5
144 lanes
The seller confirmed all four PSBs use PEX89144s.
My goal is initially to repurpose one PSB as a PCIe expansion fabric for an Alienware R12 running Linux.
The PSB already has two normal CEM x16 downstream slots. Dell's original architecture gives each PSB two host-side connections, and the PEX89144 is running Dell's signed Synthetic-mode firmware.
Initial concept:
R12 Gen4 x8 root #1 → PSB virtual-switch upstream #1
R12 Gen4 x8 root #2 → PSB virtual-switch upstream #2
Then test cheap PCIe endpoints in the PSB's existing x16 slots.
Long term I want a fabric for:
RTX 5090
RTX 3090
RTX 3090
ConnectX 200GbE NIC
NVMe
I understand the R12 would limit the upstream bandwidth; the point initially is proving the switch fabric and topology.
I’ve done extensive research into Dell's XE9680 service manuals, cable maps, PEX89144 firmware release notes, Broadcom PEX89000 documentation and the OCP HGX material.
So far I have NOT found evidence of a cryptographic/runtime Dell motherboard authorization requirement for the PEX89144. The remaining uncertainties appear to be:
1. Exact PBB/PSB power rails and sequencing.
2. REFCLK/PERST# distribution to the PEX upstreams.
3. Whether Dell's signed Synthetic firmware will initialize normally with generic PCIe root ports rather than the original XE9680 Xeons.
4. Mapping the additional PEX downstream paths that originally feed the HGX GPU tray and NVMe backplane.
The donor includes much of the original PDB/PBB/PSB wiring and cables, so I can preserve Dell's original power/control environment while testing.
I’m comfortable reverse-engineering wiring, soldering, building controlled-impedance adapter PCBs, tracing sideband signals and doing Linux PCIe diagnostics. I would not hand-wire the Gen4/Gen5 differential pairs.
There is also a fallback: the seller may have the original XE9680 system boards, 54V PSUs and other removed components. If the generic-root approach fails, I could restore enough of the native XE9680 platform and run that as the Linux compute server.
The donor chassis is currently $2,195 before negotiation and contains four PEX89144 PSBs, so this could potentially be an extraordinary amount of PCIe Gen5 switching hardware for the money.
Before I buy it, I’d really value your opinion:
Do you see any architectural reason a Dell PEX89144 PSB like this could NOT be made to enumerate behind a generic Linux PCIe root once power, REFCLK and PERST# are correct?
Have you encountered Dell/Broadcom Synthetic-mode PEX89000 switches that require platform/BMC initialization beyond normal electrical sequencing?
Would you approach this by adapting the PSB to the R12 first, or would you skip that and restore the XE9680 motherboard environment?
And if you know of documentation/tools for PEX89144 Synthetic-mode topology, Dell PSB pinouts, or PEX89000 management that I may have missed, that would be extremely useful.
I can send photos, Dell part numbers, cable maps and the exact donor listing if you'd like to see them.
Thank you,
You’ve got some code smell going on in your post that’s scaring away would-be repliers.
Like this.
1 Like
I am trying to look into this, from what I am finding I really think this would be tossing money away with no benefit whatsoever, literally better to buy an Aliexpress multi PCIe 5.0 x16 board with switching, and only one or more standard connectors to a host x16 slot(s)… Or get an old threadripper CPU & Motherboard plus DDR4.
The switch/ riser boards only seem to have two x16 slots, and worse yet is a really odd upstream connector that would go to the specific motherboard these servers were built for. There was a decent writeup I saw on “StorageReview" with photos and it seriously looks like you cannot adapt any of this gutted server unit to use with traditional consumer motherboards, especially with it missing PSUs & Motherboard, CPUs, RAM, fans, storage and much more.
Treat any part of that server assembly as not compatible with anything other than that exact server build, because I see no reason it would be anything else.
An Alienware r12 also looks like a terrible host system to try to cram a bunch of GPUs onto with what looks to be a couple of impossible to adapt server 2 slot PCIe switches for way too much money.
So no, I wouldn’t try that.