Hello! I have the usual issue everyone probably knows: All my SSDs are full. So I’d like to fix this issue once and for all and buy two 8 TB U.2 SSDs, which are designed for PCIe 4.0 x4.
Now the problem with that of course is that I don’t have 8 PCIe lanes to spare for them. I have 4 PCIe lanes free that I can use for something like this. So my questions is, what is the best way to use those 4 PCIe lanes I have to connect two U.2 drives that each are designed for 4 lanes?
I know this will result in the SSDs each only getting half as much speed as 4 lanes normally can deliver, but I intend to buy “cheap” U.2 drives that only reach 3 GB/s at most anyways. And in theory, two PCIe 4.0 lanes can reach 3.938 GB/s throughput, so going down to 2 lanes per drive shouldn’t actually hurt. But even if I would lose some speed, I wouldn’t mind, the only alternative would be to use SATA SSDs which would definitely be way slower. As long as I end up above 1 GB/s, I’m happy.
I think I need some kind of adapter card that “converts” PCIe 4.0 x4 to PCIe 4.0 x8. Does anything like that exist? I have searched a lot and couldn’t find any, but I’m also not sure if I looked in the right places. So I thought if anyone knows this, it will be people in this forum. Any ideas? Thanks very much!
so you do, or do not have open pcie slots? what MB and other devices are you using. both types of ‘adapters’ exist, but we can help fit what you need with more info.
In total my mainboard has 3 PCIe slots. The first one is used up by my RTX 4090 which I want to use with the full 16 lanes. I know going down to 8 lanes only reduces the performance by up to ~15% at most in games, but I don’t just use games, so my GPU should run with the full 16 lanes. If the second PCIe slot on my mainboard is used, the first slot (with the GPU) automatically becomes an x8 slot, so that means I cannot use the second slot at all. The third PCIe slot is x4, and I can use that one without reducing the lanes available to my GPU. So that’s where I want to connect the U.2 drives.
@alkafrazin I do not know if it supports 2x2 PCIe bitfurcation, I assume it does not. So my assumption is I need some adapter card that has a chipset-equivalent onboard for “virtually” increasing the amount of PCIe lanes, so that I can then connect two 4 lane U.2 drives to 4 physical lanes.
You need something which comes with a PCIe switch (also seems to be called a “bridge” in some places). I know some PCIe to m.2 NVMe cards come equipped with a PCIe switch in order to support 2 NVMe devices on a PCIe slot which does not support x2/x2 bifurcation, such as:
I haven’t looked into adapter cables, but I assume you can find something which goes from m.2 to U.2 to connect the SSDs you want. Maybe you can find a board which goes straight to U.2 as well and just has the PCIe switch chip, I haven’t looked for those so I don’t know if they exist, I know about these m.2 cards because I was looking into one myself.
Regarding the RTX 4090 performance in games on PCIe 4.0 x8, you are very, very wrong about losing 15% as far as I can tell. PCIe 3.0 x16 should be practically the same bandwidth as 4.0 x8 and the performance drop is only ~2-3%. Not sure about other workloads, I guess you’d have to look into those specifically, but it may very well be the case that you can just use 2 slots on your mobo and forget about other complications.
Yes, something like that adapter card that you linked seems to be what I need!
But the card you linked unfortunately only supports PCIe 3.0, which would mean my 4 PCIe 4.0 lanes would run at half their speed, which would be very sad. Do you or anyone else maybe know a similar adapter card that also supports PCIe 4.0?
Or do you know any website that maybe makes it possible to neatly search for these type of adapter cards? The Amazon search really isn’t great to find something with specific specs you need.
Regarding the RTX 4090 performance in games on PCIe 4.0 x8, you are very, very wrong about losing 15% as far as I can tell.
I was actually taking that number from exactly the techpowerup article you linked. I said “at most 15%”, because the worst case in that techpowerup benchmarks is Metro Exodus where you lose 15%. On average, it’s just 2-3%, yeah.
I do not know unfortunately, these sorts of cards only really seem to come from manufacturers I’ve never heard of so I don’t even know where to look exactly other than getting lucky with the search keywords (it’s how I found the one I linked). Still you said you’d be happy with above 1GB/s so a PCIe 3.0 switch card should still be plenty fast to reach that if you can’t find a 4.0 variant, though I understand not wanting to leave performance on the table as well.
I would suggest getting a big old hard drive and using an SSD for PrimoCache. That makes regularly accessed files as fast as SSD whilst providing massive cheap storage. The bigger the SSD the longer the files stay fast.
Yeah, it’s certainly better than nothing, it should reach up to 2 GB/s per drive. If I cannot find a better card with PCIe 4.0, I would buy that one. But if a more optimal one would exist, I would of course still love to find it. I’ve spent a few hours searching now and I cannot find any, but I assume it probably does exist somewhere, why wouldn’t it… it’s not like PCIe 4.0 is “new” tech. I can understand why there are probably none with PCIe 5.0 yet, that is still quite new.
It would be great to have such a card that can convert 1x PCIe 5.0 x4 to 2x PCIe 4.0 x4, that would allow even using the fastest PCIe 4.0 SSDs without any loss in performance. That would be a good reason for me to upgrade my MB/CPU/RAM to one that supports PCIe 5.0.
I guess PCIe 4.0 is still fairly recent in the consumer market (hasn’t even been 4 years I think) and it might not have trickled down to all these effectively no-name companies making all kinds of products to fill various little niches. 3.0 is well over a decade old and as you can see that bridge chip is still quite expensive for what it is. PCIe 4.0 equivalents might not be cheap enough to find on these sorts of cards that I would assume only few people buy.
I wonder if an alternative to “PCIe to U.2” or “PCIe to M.2” cards might be “PCIe to USB” cards, and then putting the U.2 SSD in an external USB case?
The advantage of this would be that USB can be split up into as many ports as I want, so I could even connect 4 or 6 or 8 or whatever SSDs to that card instead of being limited to only exactly two.
“PCIe to USB” cards are a lot more common, so those seem to be much easier to find.
The problem is that one USB 3.2 connection only goes up to 20 Gbit’s, so ~2 GB/s in total is just half of what even PCIe 3.0 x4 can do. So connecting two SSDs to a single USB 3.2 connection would limit them quite a lot.
But in theory there could exist a PCIe 4.0 x4 card that has four USB 3.2 20 GBit’s connections that can all be independently used with 20 GBit’s, and that would be quite nice. Or even two times USB 4 with 40 GBit’s, but that’s likely still too new.
This is the use case I’m keeping a close eye on: 4 × PCIe 5.0 lanes to 16 × PCIe 3.0 lanes for 4 M.2/U.2 drives.
Given how slowly PCIe 4.0 solutions have rolled out, I’m not expecting PCIe 5.0 solutions any time soon.
Over USB4, the sequential speed is not bad—close to direct-connect PCIe, but the latency/IOps sucks. The Optane P1600X 118GB got half the IOps (1.6 million IOps to 800 thousand IOps). There are a lot of Thunderbolt-based solutions from OWC that will let you connect a whole bunch of M.2/U.2 drives if latency/IOps is not a concern.
Over USB4, the sequential speed is not bad—close to direct-connect PCIe, but the latency/IOps sucks . The Optane P1600X 118GB got half the IOps (1.6 million IOps to 800 thousand IOps). There are a lot of Thunderbolt-based solutions from OWC that will let you connect a whole bunch of M.2/U.2 drives if latency/IOps is not a concern.
If “sucks” just means going down from 1600k to 800k IOPS, that really doesn’t sound too bad, I would be completely fine with that. A SATA SSD is usually around 100k, so way, way less.
What Thunderbolt-based solutions are there from OWC that would suit what I need? Do they have something like a “PCIe 4.0 x4 to multiple Thunderbolt” card? And does Thunderbolt actually work on my X570 AMD platform?
I can find a lot of “PCIe 3.0 x4 to 1x USB 3.2 20 GBit’s” cards, but those of course don’t even make full use of PCIe 3.0 x4.
PCIE bifurcation from 4 lanes to 2 lanes means you’ll never go over half the available bandwidth per SSD, assuming it is even supported by the motherboard and CPU (I’ve never seen it on Zen3).
Well, you see… I only have one data point. I haven’t determined if the latency/IOps penalty is a subtractive or multiplicative factor, nor have I determined if it’s a limitation of the Thunderbolt bridges in the chain. Let’s say you connected a drive natively capable of 100k IOps. Does it get halved like the P1600X or does it operate the same because 100k < 800k?
Maybe somebody else can provide some more data. Or I get more drives and Thunderbolt enclosures to test the hypothesis.
Since Thunderbolt’s nominal speed is 40 gbps (4 PCIe 3.0 lanes), I doubt there are any products on the market that would have PCIe 4.0 support on the other side of the cable.
Here is OWC’s current product line for desktop storage. A lot of their NVMe Thunderbolt enclosures work with the 3.5″ form factor, for which they conveniently have the world’s only 3.5″ U.2 device: U2 Shuttle. For the more common 2.5″ U.2 drives, you’ll need to use their holders/trays to connect the drives.
I’ll let someone else who owns an X570 setup chime in about Thunderbolt support. I’m on X670, and even there USB4/Thunderbolt support is relegated to the highest end motherboards.
Keep in mind that most Thunderbolt devices aren’t even capable of approaching the nominal 40 gbps speed of the interface no matter how fast the drive inside it is.
I’m quoting the manufacturers’ claims. Ostensibly, there is some rounding involved, because there’s an interesting blog post that explains why data transfers over Thunderbolt will never exceed 2,650 MBps, and it’s because a minimum of 8 gbps is reserved for video signals at all times, shaving 1 GBps off the theoretical cap for 4 lanes of PCIe 3.0. A PCIe 3.0 M.2 drive connected directly to the CPU can hit 3,900 MBps.
Thanks, that PCIe switch you linked is very interesting, since it actually supports PCIe 4.0! But it’s 16 lanes, and I only have a slot with 4 lanes available…
Now I wonder, what happens if I connect that PCIe switch to my PCIe 4.0 x4 slot? The slot on my mainboard does have the size of a PCIe x16 slot, but it only has 4 lanes physically connected. So I could physically connect a device like a GPU or that PCIe switch that has a physical size of x16, and in the case of a GPU I know that would actually work and result in the GPU simply running with 4 lanes. But does it also work like that with that PCIe switch you linked?
So can I plug the Rocket 1580 HBA into a PCIe 4.0 x4 slot (with a physical size of PCIe 4.0 x16) and it would still function, just with 1/4 as much total bandwidth compared to using a PCIe 4.0 x16 slot?
A slot of a large physical size (e.g., x16) can be wired electrically with fewer lanes (e.g., x1, x4, x8, or x12) as long as it provides the ground connections required by the larger physical slot size.
In both cases, PCIe negotiates the highest mutually supported number of lanes. Many graphics cards, motherboards and BIOS versions are verified to support x1, x4, x8 and x16 connectivity on the same connection.
Nice, thanks! That sounds good, I should be able to use it then.
Is the Rocket 1580 the only such PCIe switch that exists, or are there any alternatives to it? Considering it’s basically doing the same thing like the chipset on a mainboard, the price of it is quite expensive, but I guess that’s just due to it being a low volume product. If there is no alternative, I would be willing to pay that price.
Being able to connect 8 U.2 drives that all run with good speed (as long as they aren’t used at the same time) is very nice for “future proofing”. Now I just wish it would support PCIe 5.0 already!