New NAS build (expandable) (Seeking Advice)

[Noviets]

Hopefully this is the right place to post this.

I recently put together a 3900x server that was going to be used for work, but we ended up going with aws instead. So I moved my plex server over to it.
I have a current nas of only 4 drives (QNAP, this is where I moved my plex server from), but it’s quickly getting full (only used for plex media).

I host the server on the 3900x server (running Ubuntu) and have mounted the nas on the server to use the media library. Everything working well (except the need to automatically scan when I add files as it doesn’t auto detect them anymore like it did on the QNAP).

Intention:
Rather than getting another nas and having this problem again, I am looking at getting a 24 drive 4U chassis and building my own “disk shelf” of sorts (except with an atx power supply and 120mm noctua fans so it’s quiet, as this will be in the office next to me where I work).

If I get 24 drive chassis, I am looking at going ZFS and having 4 drive vdevs, So I can get 4 more drives, move my current data over to it, then format my existing raid 5 on the qnap nas, and have 2x4 drive vdevs on it to start.

Then I can just more them in sets of 4 drives going forward.

Questions:
Basically I am looking at getting some help on what I need to buy in regards to the IO, I’ve not built my own serious storage before and have not used SAS, LSI cards etc. So not sure how the server connects to each of the drives.

Cards in the system using PCIe lanes are: a 10GBe network card, graphics card and an m.2 drive, so I should have enough pcie lanesif I put graphics card in the bottom slot, (I added it for transcoding, so I could likely get away with it being x4 for example, or removed entirely if determined unnecessary).

I guess my first step would be finding a 4U hot-swappable 24drive chassis?

Sorry for the long post + story, trying to learn this stuff, gotta start somewhere

[zlynx]

@Noviets

I guess my first step would be finding a 4U hot-swappable 24drive chassis?

Sounds like a good place to start.

I’ve only seen a sort of proprietary Dell drive chassis like that. It was SAS and had a special cable which I think was just 8 SAS cables bundled together. It did make it convenient to plug in.

Edit: I was wrong about proprietary. See below, I was remembering these SFF-8644 connectors I think.

You are going to want SAS and multiple cables (even if it looks like one cable) because each SAS port and cable can do 12 Gbps, but I think doing more than 4 drives on one port is a bad idea. I think the protocol supports up to 128 drives per port, which just seems ridiculous.

[zlynx]

@zlynx

but I think doing more than 4 drives on one port is a bad idea

Huh. HP at least disagrees with me. I don’t know what they’re thinking but the HP drive racks seem to use single SAS cables for loads of drives. I don’t see how they expect performance to be any good.

Edit: I was wrong about this but leaving the post anyway.

[zlynx]

@zlynx

special cable which I think was just 8 SAS cables bundled together

Ah shows what I know. I think it was these things: https://cs-electronics.com/sff-8644/

They get 4 12 Gbps SAS links into a single cable. Two cables for redundancy and you get 8, which is probably what I was remembering.

[gordonthree]

It’s likely SFF-8088 which provides 4x 6GB or 12GB SAS channels (ports.) Most configurations would use two of these. That’s plenty for several shelves full of spinning rust or even SAS SSDs.

Look around for a used SAS shelf, like NetApp, HP D2000, etc.

[Dutch_Master]

Here’s a candidate for that chassis: Inter-Tech IPC 4U-4424
A large German retailer (<-link!) offers it at about 485 euro. Maybe you can get it for better price on certain well-known sites Quick summary: 1x 3.5" or 2x 2.5" (with adapter) internal drives, 24x hot-swap 3.5" drive bays on front, no PSU but room for an ATX unit. Can hold up to 7 PCIE cards. Supply your own mainboard., of course.

Apropos mainboards, I can’t decipher whether or not you transplant the 3900X system or start from scratch. In case of the latter, consider the MSI MEG X570 GODLIKE mainboard. Quite expensive, but with a ton of useful features, like both 1 and 2.5 Gbit network, up to 128GB RAM, 6 (or even 8!) SATA ports and 3x M.2 slots as well as no less then 4 PCIE 16x sockets. And it’s an AM4 board, so you can use a reasonably priced proc. You may have to if your budget is tapped out by buying that mainboard

[edit: the Broadcom SAS 9305-24i offers 24 SAS ports in a single PCIE card. Not cheap though!]

[zlynx]

@gordonthree

Look around for a used SAS shelf, like NetApp, HP D2000, etc

And watch out for the details. Used stuff does not always come with cables or drive trays or other important things.

This one used computer recycling place in Phoenix sold almost free old dual Xeon servers with 512 GB of RAM but charged $50 each for the drive trays.

[zlynx]

@Noviets

Rather than getting another nas and having this problem again, I am looking at getting a 24 drive 4U chassis and building my own “disk shelf” of sorts (except with an atx power supply and 120mm noctua fans so it’s quiet, as this will be in the office next to me where I work).

I just reread your post and realized there’s confusion on my part about what you are looking for.

I was thinking you wanted just a drive rack. Just drives. No CPU.

But then I realized that you might mean a rack server with CPU, RAM and drive bays all together. If so you don’t need weird external SAS cables and ignore what I wrote above. Heh.