I have been working with NetApp gear professionally for about 15 years now, so I have gathered some old equipment over the years…
I have a few older disk shelves for 3.5" drives, DS4243 as they were originally known for… the 3 at the end is for 3Gb (the first 4 is for 4 rack units…).
They come equipped with two IOM modules at the back and they come in three different flavours. IOM3 = 3Gb, IOM6 = 6Gb and IOM12 = 12Gb.
The two first modules uses QSFP connectors, and the last uses MiniSASHD.
I know that the Q in QSFP is for Quad, so there is actually four times 3/6/12Gb.
Now the shelf is just a SAS expander in nature… but does anyone know how it is actually connected inside? Are the four connections split into 6 disk pairs?
Or are all drives connected to one large SAS bus and the Quad connections is just somehow shared between them?
The reason I ask is to be able to better utilize the bus, because if you have 24 12Gb drives in there that theoretically can operate at say 250MB/sec. which would be 6GB/sec. which is in fact the maximum speed of 4 x 12Gb/sec.
So that should be OK then?
Other than NetApp supports no less than 7 daisy chained shelves in a loop…
Of cause this is in a redundant setup where one controller has two connections (there are two IOM modules in a shelf). And if in a HA setup the other controller also shares the bus with another two connections…
7 shelfs at 6GB/sec. will never be able to get their data though four 12Gb SAS HBAs in the controllers…
And yes, this is all very theoretical in regards to speeds
But back to the shelf… if I should guess, the shelfs SAS lanes are divided up into four lanes which are split between the four rows of disks in the shelf.
That would make sense in my world.
Unfortunately it is hard for me to confirm, even with deep debugging on a NetApp system… I can only see the four SAS lanes link up, I cannot see where the individual disks link up. I also have a ZFS setup with a DS4246 and also I cannot find anything that points to which lane the disks are connected.
So I guess this is abstracted in the IOM module somehow?
If anyone knows more about this, I would be interested in knowing more about this…