Noob question. I have a PCIe - nvme adapter. Does it provide PCIe v4 speeds if I put a v4 m.2 card in or does the adapter itself determines / limits the protocol version / transfer speed? From the pcb it appears to have a couple of lanes and only few electronic components.
Depends. Negotiated PCIe speed depends on 3 factors:
- supported gen by controller/host
- supported gen by endpoint
- link quality*
Your adapter influences the 3rd factor. If 1 and 2 are gen 4 then it’s possible that they negotiate gen 4. However based on actual signal quality it may drop to 3 or even 2 or 1. It may even be possible they negotiate 4, but due to transmission errors actual speed drops. Same as with cables.
4 Likes
Nice surprise: The Delock 90504 can handle any type of PCIe Device, not just PCIe NVMe SSDs. Could make even more jankier builds possible 
1 Like
This to the currently stupid cheap 9500-8i? I just picked one up for a dumb test build for like 60 bucks on ebay. It would be really funny.
The PCIe Gen3 Switch parts have come down in price nicely. The Broadcom HBA 9500 line has the advantage of having a PCIe Gen4 host interface reducing the needed amount of PCIe lanes for similar performance and of course giving you the option between usding NVMe, SAS or SATA drives.
Personally for NVMe SSDs I’d prefer a pure PCIe Switch and not a Tri-Mode HBA but when wanting PCIe Gen4 or faster these are either bugged (Broadcom P411W-32P) or pretty, pretty pricy (the stuff from C-Payne).
Yep. Pretty funny solution to think about though, right? hahah. Btw, have you seen this listing?
I bought one and need to do some writeups on here soon when I get everything going. The catch is that the power connector in the top middle is not eps12v, its microhi. Other than that, it’s pretty damn cheap for what it is, don’t you think?
1 Like
I don’t have any experience with DIYing UBM backplanes yet and am looking forward to your findings!
-
Be sure to update your HBA 9500’s firmware to the latest version before doing anything with it to avoid unnecessary frustrations.
-
Are you able to measure the UBM backplane’s (idle) power draw on the special power connector without any drives in it?
-
Could you please test how the complete configuration (HBA 9500 + backplane) reacts when sending the entire system into S3 sleep (suspend to memory) and waking it back up again?
I don’t either. Is this a ubm backplane? I am not sure what qualifies. It has 2x slimsas 4i to handle 8 sas drives but also 4x mcio 8x. I am not sure if this disqualifies it as ubm backplane. Would it need to work with nvme over the slimsas ports for this to be a ubm backplane?
- Yep, already know lol. Was bitten by that once already with a 9400 not working with a sas expander until I updated it (there were no guides except one for lenovo, helped flesh out the procedure for the regular 9400)
- I sure can. It probably won’t be that much tbh. I just need to wait for the cables to come in. Do you want me to @ you for the writeup on the crap I have had to go through to try to find stuff for that microhi connection haha. I am still looking for a direct adapter, but since I can find none I am simply going to solder myself one. The cables are in the mail right now, hopefully I will have the time to do it when they get here
- Sure. How would I do that on command? Also, would it make a difference if I do this with my regular tower vs my xeon (2660v3) testbed, and if so, should I test that with both?
1 Like
I’ve been looking for a 4U case that can hold many U.2 drives and this seems promising. Did you manage to use this in a non Supermicro chassis? I had the impression that this would only work in the specific servers that its meant for.
Does it run at PCIe 5.0 speeds? What are the cables that I’d need to connect it to the motherboard and power?
On the PCB of the actual backplane it says “NVMe/SAS”.
-
SAS and NVMe are entirely different protocols using different electrical interfaces, so the backplane has to be intelligent to determine what type of drive with its generic SFF-8639 U./U.3 connector is plugged into it. Otherwise it might physically damage components.
-
To my knowledge as a lay person that means that it’s an UBM backplane.
-
Passive backplanes that are just compatible with Tri-Mode HBAs (like the Icy Dock ToughArmor MB699VP-B V3) can exclusively be used for the type of drive they are designed for, in this case PCIe NVMe SSDs, not SAS or SATA drives, even if the connected Tri-Mode HBA would be able to talk to any type of drive.
-
I regularly visit this thread, so I’d be very thankful for you just posting your experiences with that SuperMicro backplane here. This thread is intended to be a knowledge base for everything around PCIe so these experiences would be highly appreciated here.
-
I can’t say it for sure but since that backplane has active components to negotiate the kind of drive inserted into it (not just PCIe Gen3/4/5 but also SAS/SATA) I’d say that’s 99 % certain that PCIe Gen5 SSDs will only operate with a downgraded PCIe Gen4 interface.
-
I’m pretty sure that you can’t just passively adapt this backplane into a generic PCIe Slot, you need a Tri-Mode HBA or PCIe Switch (for example the Broadcom P411W-32P) where the firmware is able to negotiate what kind of interface is being used. Of course, with the P411W-32P only PCIe NVMe SSDs would actually work.
Hmm… Are you sure about that? A fully passive backplane should be indistinguishable from a fully passive cable, and AFAIK the cables are compatible between the three.
I have all three current variants of the Icy Dock ToughArmor MB699VP-B: V1, V2 and V3.
-
V1 is in my opinion falsly advertised to be able to do PCIe Gen4 (look at their website), it introduces PCIe Bus Errors like there’s no tomorrow when actually using it with PCIe Gen4. PCIe Gen3 is fine.
-
The box the V2 shipped in is used for the V1 as well and it’s checked if there is a V1 or V2 in it. And behold: Next to the V1 it says that it’s only intended for PCIe Gen3. I’ll never forgive Icy Dock for that since they haven’t acknowledged this.
-
The V2 works with PCIe Gen4 without PCIe Bus Errors, you can hook it up directly to a motherboard’s PCIe lanes or PCIe Switches like the Delock 90504 or Broadcom P411W-32P with old firmware.
-
Newer firmware blocked the P411W-32P from being able to be used with directly attached SSDs (you now HAVE to use a “proper” UBM backplane) and with that newer firmware it stopped working with the V2 backplane. Reverting to the older firmware restores functionality.
-
Tri-Mode HBAs (tested Broadcom HBA 9500-16 and an Adaptec HBA 1200p-32i) don’t work with the V1 or V2.
-
The V3 backplane works with everything, generic motherboard PCIe as well as PCIe Switches and Tri-Mode HBAs BUT as mentioned not with the P411W-32P with the latest firmware that requires there to be an UBM backplane.
-
SATA SSDs remain dead in the V3 backplane while it’s connected to a Tri-Mode HBA. That observation fits with Icy Dock’s statement:
*Note - The MB699VP-B V3 only supports NVMe SSD when connecting to a Tri-Mode controller. It does not support SAS/SATA SSD/HDD.
- I don’t have 2.5" SAS drives so that’s the one configuration I can’t verifiy.
2 Likes
Try Rocky Linux 8.10 or 9.4. I find higher performance with NVMe drives over all Ubuntu LTS releases (18.04, 20.04, 24.04)
Sorry, been too busy to respond lately.
Ive had many, many different backplanes that dont have ubm and work with sas, sata, and nvme, all with the same connector. Some of these go back to 2017 or even earlier, before ubm was ever a thing. Even before ubm, I have never encountered any danger or damage with any of my drives in these kinds of slots, and highly doubt I ever will. I don’t believe that they would be so dumb as to allow drives that fit into the same slot to be nuked, but I could be wrong. If that indeed is a risk, do you know with what drives and what machine/backplane so I can test that? An example of a backplane before ubm that supports these is the BPN-SAS3-216A-N4. The rightmost ports are sas/nvme (doesnt list sata but iirc those work fine too).
I’ll let you know how it goes. I had to purchase some more cables to get it to work. The power cables are here, and I need to find a way to adapt from something I can use. Probably means soldering lol. I need to document these connectors and standards, though, as zero resources exist for microhi, minifit, and possibly one or two more. It was 80 dollars just for those power cables, though, and finding out more about these may require more emails or buying more cables. Ugh. The controller, an mcio adapter card, and mcio cables are on their way. They are going to take a while though, so don’t hold your breath. The ebay estimate said latest arrival was november something lol.
Will do. I’ll @ you when I do all this stuff. You’re one of the few weirdos that, similar to me, is interested in this kind of pcie connectivity lol. It can be fun, or it can be broadcom 
Nah, when I get it up and running it should fully support gen 5 ssds without errors. I got that cheap pcie to mcio adapter and the cables off of ebay, so who knows how well that will work haha. Its not like I have any gen 5 ssds though, I just found that this was a fun way to get u.2 gen4 drives to work with a backplane without spending nvme4 money. I suppose I am failing at that, though… lol
It goes straight into the board for their servers, just as mcio. Maybe the board is doing that kind of negotiation but I dont think that it is since there is a decently sized chip and some logic on this board.
I can send you a godawful sas ssd to play around with if you want to give it a shot lol. I don’t remember the size, might be 240 or 480 and its one of those early brick-like ones.
Haven’t hooked it up yet. Might be a long while before I am able to give them a test. I have no gen 5 drives or system to test this so I wont be able to test g5 at all. If you actually want to get g5 drives working with u.2 drives, then get a supermicro board and buy their straight mcio 8i to u.2 cables and don’t look back. They sell them direct from their shop. Set it to bifurcate and bobs your uncle. Let me know if you want a link or the part number. This is the easiest and best way to ensure that you get working gen 5 u.2, and is uncomplicated. If you have a different board with mcio 8i or 4i it should work as well given the right cable. If you want to use a board without mcio or one with just slots, then I have some bad news for you, since adapting to g5 u.2 is either not cheap or is not tested or assured in any way. It’s extremely difficult to know without buying and testing all of these stupid things. @aBav.Normie-Pleb if you have g5 and want to borrow all this garbage to test (or to test with the p411w-32p) then I would be overjoyed to lend it all to you. That would mean way less work for me haha. I just need to modify one of those new power cables that I just got to adapt from eps12v 8p female to microhi 8p male lmao. We’ll see how that goes when I get those mcio cables, after this ebay guy responds to my inquiry at least…
2 Likes
Sure, could you drop a link to the Supermicro part?
I’ve been thinking about getting an SP6 motherboard to use with Epyc 8004 for a lower power platform that still has enough PCIe lanes for expansion. The motherboards I’ve seen all have MCIO ports.
I asked about chassis compatibility for the backplane you mentioned because I was looking for a 4U case (wanted more space for part compatibility and quieter cooling) to use for the build, and was hoping that backplane may simplify connectivity to multiple U.2 drives. It seems my best option may be to run cables from each MCIO port to 2 U.2 drives directly without any backplane.
There are others out there, but this one is absolutely sure to work, at least with supermicro boards. 30-40 dollars per u.2 drive connected is pretty standard, sorry.
- CBL-MCIO-1245U2Y-E - 80 dollars
The backplanes are more for if you’re dumb (like me) and want things all in a neat little row. It does have some small benefits, such as compatibility and support for multiple drive types depending on the backplane, in addition to some amount of forward-looking compatibility, but in general the benefits are rather minute.
A novel PCIe switch in a U.2 form factor I just got: https://forum.level1techs.com/t/post-what-new-thing-you-acquired-recently/149881/21352
I haven’t thought of a use for it yet. Each M.2 slot gets 2 lanes from the PCIe switch. Apparently, the switch has 24 lanes in total and each slot gets 4 lanes.
Holy crap that is cool. That’s amazing. I wonder how it handles the power consumption of all of em combined + the switch chip?
I put an Optane SSD 800P on it, and it takes the entire contraption right up to the 15 mm height limit. There is barely any room for a heat spreader on top of the SSD inside the U.2 carrier. I haven’t tried yet, but I reckon it has to be less than half a millimeter to avoid warping the SSD or the carrier. I have no idea where to get one that thin.
The switch chip is also interesting in that it’s hypothetically configurable using a command line tool (switchtec-user), though there are neither documentation of nor example configuration files I can use as reference. The sell sheet teases that it can bifurcate down to 2 lanes. I’d like to know if this could be the thing that’ll make the Optane Memory H20 finally work.
U.2 get either powered by the PCIe slot they are mounted on or via a standard SATA power connector (Cables have Y-shape, so U.2 gets both PCIe signal and SATA power). SATA power is plenty to fuel a couple of M.2. I think they’re ok to up 50W