Am I worrying too much about "lanes?" (question about M.2 drives)

Hardware Hub Build a PC
1

TL;DR version is this:
On this new build I'm gonna be working on, I'm considering throwing an XP941 into an MSI X99S SLI Plus (a la Pistol's X99 upgrade) but I'm having some issues figuring out if that motherboard is gonna be sufficient for the features I have planned.

Details:
Relevant parts are as follows:
--> X99S SLI plus (budget is a factor; I could save up for a more expensive mobo if it's absolutely necessary but I'd really rather not.)
--> 2x GPUs (will be running at x8/x8)
--> 28 lane processor
--> an ethernet expansion card at x4 (it's sort of a long story why I'm set on this one, but basically I need MAD ethernet. Virtual setup + network bonding + blah blah.)
--> XP941 M.2

AT FIRST I just thought that I could throw in the GPUs,(x8/x8) chuck in the ethernet card,(x4) and then toss in the M.2(x4) and everything would be gravy, because I'd have only 24 lanes taken up with 4 left over, right?
But when I look at the specs for the SLI plus, here's what I see under the "expansion slots" section:

4 x PCIe 3.0 x16 slots, support up to 3-way mode

1-way mode: x16/ x0/ x0/ x0
2-way mode: x16/ x16/ x0/ x0, 16/ x8/ x0/ x0*
3-way mode: x16/ x16/ x0/ x8, x8/ x8/ x8/ x0*
For the CPU that supports 40 PCIe lanes

** For the CPU that supports 28 PCIe lanes

When I see that data, what I'm now thinking is that this mobo can only use three slots at one given time, even though it physically has four slots.
But is the M.2 even affected by this "up to 3-way" limitation? Because it doesn't look like it's even physically attached to a PCI slot. And this then leads me to believe that the M.2 may have a logical connection to one of the PCI slots, in which case, "up to 3" would mean there's one device I wouldn't be able to use. Right...?
I've googled around for a bit but I can't seem to find a solid answer as to how the M.2 would interact with the PCI bus, whether or not it's eating up lanes, and I've seen some more confusing stuff about the "CPU having different lanes from the chipset" or some devilry like that.

So basically, am I worried over nothing or do I need to rethink my build a little bit?

2

Alright, I've been patient.

Am I getting ignored because I broke the rules somehow or because nobody knows the answer?

3

no idea, i would say. Because i certainly don't have one :(

but i would like to learn :D
how does the lane counting work?
Is it maybe correct when i say:
You've got 4x16 Lanes maximum. So you'll be fine. Because this up to 3way only means 3 GPUs maximum at a time?

4

Here's the deal...

I have a similar setup, using the SM 951. Assuming you are plugging the M.2 into an M.2 slot, rather than a PCIE slot? If so, you will probably have no trouble. Best bet is to check the motherboard manual online first. I'm using the Asus X99-S, which has this written in the manual:

*1: The PCIe x16_5 shares bandwidth with M.2 x4. Triple PCIe 3.0/2.0 configuration is default set at x8/x8/x8. Adjust PCIEX16_5 Slot Bandwidth in BIOS.

So I am not using the 5th PCIE slot because I'm using the m.2

Also, if available get the SM951.. roughly the same price and quite a performance jump. I'm talking about the AHCI version of course, the NVME is about double the price right now.

The manual also stated this:

*2: The PCIex4_1, USB3_E12 and SATAEXPRESS_E1 connectors share the same bandwidth. The SATAEXPRESS_E1 will be disabled when there is a device installed on PCIEX4_1 slot. Set this option to X2 Mode or X4 Mode when the installed PCIe device is higher than X4 interface.

This may be different for you of course, but it means I have to pick between those 3 things (which is fine for me).

All I can say is, keep your eyes open. If in doubt and you have the budget, buy a workstation board with extra PCIE lanes and all the trimmings!

1 Like
5

Lanes are determined by the CPU (99% of the time -some workstation boards provide extra through a PLX chip). In this instance, the OP is looking at what I assume to be the 5820k, as it is the only CPU i know of which has 28 pcie lanes.

Graphics card will take up 16x lanes (But can be changed to 8x as you get 95% of the performance, even with a titan x)
m.2 will take 4x lanes
Secondry graphics card will take up 8x lanes
ethernet expansion card will take up 4x lanes

total: 24 lanes (assuming he gets the first card to work as 8x)

1 Like
6

thanks so much!

7

considering when when you posted it many people where asleep.

And Yes you will be fine with two GPUs and a M.2

8

Looking at the PCIe Bandwidth Table included in the manual, it appears that the top 3 PCIe 3.0 lanes will be set to x8 with all 3 slots populated and a 5820k installed. 4 PCIe 3.0 lanes will be reserved for the M.2 slot. However, the XP941 doesn't use PCIe 3.0. It uses PCIe 2.0. If you use a XP951 ssd (which is PCIe 3.0) you should have all the PCIe 3.0 lanes allocated and still have 2 PCIe 2.0 slots available but they will be blocked by the graphics cards unfortunately.

1 Like
9

Yeah, that was actually exactly what I needed, thank you.
That makes it easier, since I only have to worry about X number of slots-- initially I thought I'd have to factor in a bunch of weird abstractions.

Anyway, thanks again for the advice.

10

That's good to know.

I'm a wee bit unclear on what you're saying, though: I'm aware the XP941 uses 2.0, but I was under the impression that if I threw a 2.0 device into a 3.0 slot it would still work, just at 2.0 speeds or something. Is that inaccurate?

Thanks again to everyone who responded.

Sorry to say I don't quite understand your last sentence, but basically how it works (I'm just paraphrasing PoshGeordie up there) is that your processor determines how many PCI "lanes" you can use at once. (if it's AMD, the entire chipset determines it, regardless of the cpu.) In my case, I'm using a 5820, (spot on, Posh) so I would look at Intel's official page:


( ^ Normally if you google the chip's name, this datasheet will be the first result.) At this point I just look at "Max # of PCI Express Lanes," and see that I've got 28 to work with.

That means I can't use two GPUs at x16 each, so I have to kick them down to x8/x8. (they barely lose any performance.) You could also look at your motherboard's specs, and if you look at the data in my initial post, some of them only support up to a certain number of PCI devices. (that number can also change if you have a different processor, as indicated by:

(My only confusion was regarding how the M.2 fit into this; I'm still a little foggy on how those work, but as Knight has pointed out, there oughtta be specifics in the motherboard manuals, which you can look up online.)

Sorry for the wall of text; hope this helped, though.

11

I could clearly see 20 views within the first two hours, and my second post was worded only as to ensure I wasn't violating any ToS or something like that.

And it's two GPUs, an M.2 as well as an ethernet card, good sir.

12

Since i didn't know anything about the matter my sentence probably didn't make sense anyway. thanks for the explanation of how this works! I never considered stuff like this when building PCs.

13

You are correct. The 3.0 lanes are backwards compatible with 2.0. I was just pointing out that those dedicated 2.0 slots would be blocked by the graphics cards so they would be unusable. Leaving only one expansion slot left, PCIe_E6 will be deactivated with a 5820k installed.