Good high-end NAS cases don't exist, Should the community make one?

Hardware Hub Cases / Chassis
41

Several reasons for not using usb3.
usb can only connect to one device at a time, and has to disconnect from the first device to talk to the next device. It can do this at like 100 times a second, but we would prefer better performance than that.
USB only performs blind writes, ie the usb communication bus cannot confirm whether data was written to disk or not. If one of 8 drives in a raid array was bumped, you may never know until you attempt to retrieve the data that data is missing.
Most USB connectors are not latching. Almost all SAS connectors are latching, especially the external ones. Think of ethernet, if you tug on the cable it won’t fall out of the hole.
16 channels of SAS on one card is common. Do you know of a usb3 controller card that has 16 channels? Some usb cards are 2 channels, with internal hubs to 6 external ports, but those are still only 2 channels.
On that syba box, if it had 8 sas channels, it would be at least 4 times faster, and potentially hundreds of times faster.

Your typical hard drive goes around 200 megabytes per second, or 1.6 gigabit. Even though the syba enclosure is 3 times faster than that, it can only address one drive at a time. If you put all 8 drives in an array, it would need to talk to #1, then disconnect, connect to #2, then disconnect, then connect to #3. If a read event occurred on #1, that read event would need to complete in order to write to disk #2.

If you had one of the $35 8 sas channel adapters like:

, and one of the $150 24 port sas expanders, like:

You would have a computer with 8 connections to any of 24 different hard drives.
The way that this would likely be organized is with a zfs array. an M.2 ssd would be the “slog” and “L2ARC”, there would be a single pool, organized as 3 raidz2 vdevs (virtual devices) each consisting of a set of 8 hard drives. Write events would go to one set of 8 hard drives via all 8 the sas channels simultaneity for large writes. Smaller writes would only go to a smaller set of drives. Read events would go to the drives that contained the data being accessed. Often this would be a set of 8 drives. When 8 drives get accessed at 200MB/seconds, the transaction gets processed at 1.6GB/second. While this is not very fast in comparison to common SSDs, it is much faster than most single drives, and usually less than 20% the price of most SSDs.

With the above knowledge, use 2 16 channel cards like:

4 sas expanders like the ones listed above
and you get a fairly performant storage server, and you are only using 16 PCIe lanes, so you can still use the computer to perform actual work too.

1 Like
42

Where did you get this idea from?

43

I read the spec for version one of USB.

is it different now?

apparently it is.

USB now allows for “Isochronous Transfers” and “Bulk Transfers”.

“Isochronous Transfers” are used for audio devices, mice, and keyboards which require a small amount of data infrequently.

“Bulk Transfers” are time insensitive, and fit in when other transfers are not occurring.

However every data transfer has a setup and teardown phase, and transfers to multiple devices cannot occur simultaneously. It can just fake it due to having a fast enough transfer rate. This is assisted by having the maximum packet size scale with the bus speed, so that latency remains low. With a busy bus you will still see latency though, and the solution is to use a different port for higher bandwidth devices than for latency sensitive devices.

From the referenced link:

  • The maximum data payload size for low-speed devices is 8 bytes.
  • Maximum data payload size for full-speed devices is 64 bytes.
  • Maximum data payload size for high-speed devices is 1024 bytes.
44

Another thing of interest when looking at which bus to use…
For instance with 20tb ironwolf nas hard drives from seagate, they have a max transfer rate of 2.2 gigabit, or 285MB/s. This is less than half the speed of a 5g usb line or a 6gbps sas connection.

Per the wikipedia sas article, the common speeds are:

  • SAS-1: 3.0 Gbit/s, introduced in 2004
  • SAS-2: 6.0 Gbit/s, available since February 2009
  • SAS-3: 12.0 Gbit/s, available since March 2013
  • SAS-4: 22.5 Gbit/s called “24G”, standard completed in 2017

12gbps is fairly common, and newegg has one 24g controller. When data is transferred from a host to a drive, it is transferred at the speed of the slowest link, so the drive, controller, expanders, and all cables have to be capable of a given speed in order to take advantage of that link speed. So while a given large write may be occurring over all sas channels, other transactions can still continue to occur over all of the links.

45

USB is a complete non-starter for any kind of important data for me. There always seems to be something that goes wrong during prolonged use, be it corruption intermittent connectivity or it completely dropping out due to some driver on the host system.

Regarding the SAS bus infrastructure running at the slowest link, this isn’t necessarily the case anymore. Starting with some SAS3 expanders, there is a bandwidth aggregation technology called DataBolt that will allow a 12Gb/s expander to aggregate connections to a bunch of 3Gb/s and 6Gb/s drives into a 12Gb/s signal back to the controller (assuming the controller is SAS3).

SAS4 is good stuff, it has much higher signal integrity than nvme but isn’t quite as fast. I’ll be using it for a write cache drive in my setup.

2 Likes
46

USB 1.1 was like more than 25 years old. It’s like you were talking 1970s computer tech in 1990s.

Yeah, I don’t mind to read ppl with tech bias. But over exaggeration or down right misinformation is not reasonable discussion. :grin:

Also USB attached SCSI protocol has been in place for a long time. My cheapo Chinese brand USB 3.0 enclosure even supports it. Latest USB 4.0 will have 80Gbps transfer rate. Packed with USB PD, USB is going to stay and will have a lot more applications regardless how bad a start it was back in 1996/7…

The most popular NAS in consumer space is between 2 to 4 bays where USB enclosure is a terrific solution IMO.

3 Likes
47

While writing that I was learning and changing my opinions. Later sas versions are even better than what I was coming to understand. I was trained on sas 1, and people were just transitioning to sas2. The 1hdd per channel of simultaneous use was still the best use case. It sounds like this annoyed some bean counters, and some engineers, who fixed it for version 3.

It looks like you could simultaneously access 4-5 Hard Drives from one sas channel. Does that match your experience?

48

Oh sure, USB enclosures isn’t the BEST of the freaking BEST.

However, for the use case we are discussing, where you are not primarily concerned about performance but just need a way to conveniently set up a shitton of drives, it is good enough.

As for USB being limited - yes, it is. But not as much as you think. A modern NAS enclosure can easily juggle 10 SATA drives, and while transfer performance might not exceed 10x200 MB / second (more like, 5 Gbit/s or 625 MB/s), it is fast enough to saturate a 5+5 RAID10, for instance.

As for addressing every single drive - yes you could but normal procedure with a NAS enclosure like that is that the entire drive is set up as some sort of logical volume. In other words you instruct the RAID to do stuff before you use it, and it will have some logic to self-heal. It all depends on how the firmware is programmed, but it is safe to say these USB cabinets are getting better and better at it.

Sure, it doesn’t beat a rackmounted 60-drive monster NAS with dedicated controllers for everything. For this use case though… It doesn’t need to.

1 Like
49

Isnt that a DAS though? :stuck_out_tongue_winking_eye:

1 Like
50

Oh, sure it’s.

Just I said in another thread on another day. DAS will work so well for home use/personal computing. I have little idea how NAS could have taken off at home.

While ‘taking off’ sounds like a lot, in fact the market for home NAS is no bigger than Nintendo Switch I believe…And the majority of those sales are between 2 to 4 bays.

51

Adding to this… We’re doubling NVMe storage capacity for the same price every two or three years right now. 4TB NVMe drives are now affordable (but not exactly dirt cheap) at $229, and 8TB SSDs are getting there at $500, only a matter of time before NVMe matches that.

Three more doublings and we will be able to buy 32 TB of storage for $200. When will that happen? Given current trajectory, somewhere between 2028 and 2032. I therefore think a 4x NVMe NAS box that you can hang next to your router or switch on the wall, will eventually be the norm for the consumer space at home (think something that is 20x100x100 with two 10 GbE network ports and a USB C power outlet).

Pro will go a different route with 100+ TB 3.5" drives though:

Note that the above is all speculation at this point, I am doing educated guesstimates but nothing is known in advance. I am personally convinced that the HDDs last bastion will die this decade, but I am just another moron with an opinion on the internet so feel free to point and laugh. :slight_smile:

52

For peak performance it is still best to have 1 dedicated SAS channel per drive so that you don’t starve (more like put on a small diet) the hdd’s queue. SAS drives have a relative to SATA (256 vs 32 command) deep queue depth so they are less affected by the multiplexing behavior that expanders bring than one might think.

I think it is reasonable to have a 4 to 1 over subscription of SAS drives to SAS channels since SAS channels are getting so fast, this requires an expander though.

1 Like
53

I’m not very good at long term prediction. Though sometimes I got it right…either by luck or coincidence. One recent example is Metaverse. For the life of me I couldn’t figure out why it’ll take off, and in what way it’ll promote productivity in societies and benefit the progression of human civilisation. Faster than I thought Facebook had a second thought of its metaverse strategy after a very short lived fanfare, and at the down of seemingly eruption of AI applications.

I believe you’re right SSD/NVMe will increasingly become ubiquitous and dominate in consumer space as the primary storage medium. But I have doubt that HDDs will be going away anytime soon for consumers. Certainly very unlikely in the next ten years. Because we haven’t seen a replacement technology which is cheap and massive in capacity. So for every TB of SSD/NVMe a consumer buys, he/she will probably have to buy 3TB of HDDs for data backup purpose.

54

Ah, but here is where you are not seeing clearly. Today, consumer PCs no longer have any use for mechanical drives, or SATA interfaces. SATA is legacy and if it stands between SATA and another m.2 port I think most will opt for the later, after all you can always repurpose m.2 to 6xSATA interfaces.

Bulk storage is the final bastion of HDDs, and here I think they are about to be overtaken by NVMes, too. Let’s just crunch the numbers. So, let us assume by 2025, 8 TB NVMe for $200 is the norm. Right? Well, an 8 TB mechanical drive today costs $119. Let’s assume the value of that goes down to $79 by 2025, and $69 by 2027.

Let’s assume by mid 2027, you can get 16 TB drives for $200. For HDDs, 16 TB costs $249 today, which means they would need to drop to below $115 to be worth it. If it costs $130 then that is just 50% premium for a much faster, more reliable drive, so has to be at least 75% premium, hopefully more.

Further, let’s assume by 2030, you can get 32 TB drives for $200. For HDDs, 32 TB costs… Well, they aren’t out yet, actually. Biggest HDDs you can buy today is a 22TB for $400. But extrapolating from that, let’s say $600 for a 32 TB drive by current standards. This means this drive must have dropped 66% of it’s price tag to be even on par, and 80%+ to be competitive with the NVMe drive.

Do you think we can get 30+ TB drives for $115 or less by 2030? Otherwise, it is inevitable that NVMe will take over bulk capacity too. Only a matter of time now. I personally think the breaking point will be when 16 TB reaches $200, but I could be wrong here. HDDs are also becoming more and more unreliable for everything but sequential writes with new tech like SMR…

But I fear we have strayed from the original subject here, it is a tangential discussion that merits it’s own thread if you wish to keep discussing it. Sorry OP for derailing!

55

hahaha it probably does deserve its own thread because I have thoughts on this too; I just finished watching the OpenZFS Developer Summit talk on multi-actuator drives and some interesting points were brought up about future storage paradigms.

​​​ ​ ​

This is a valuable comment;
When I started this thread I thought I could keep the costs down via minimalism, but there is a lot of extra complexity and new features I want that I hadn’t thought about at the beginning.

What I have been keeping in the back of my mind when designing is to design the parts so that a 3d printed part could be substituted for a machined aluminum part. My thinking is that the “bones” of the case could be purchased for an aesthetic, sturdy starting point to add on to at a “low” cost.

For example one could go the route of getting all the machined aluminum parts, a 30 bay drive cage, fancy switches/front bay door, or get just the “bones” of the case and 3d print the less stressed parts and maybe only have a 3d printed 8 bay drive cage hanging in the bottom (there is no way I would trust a 30 bay drive cage to fdm 3d printing structurally).

I know we’ll never be able to compete on cost with the big manufacturers that do volume, but we can come up with better features and outsource some of the value creation to the consumer by having them construct their own pieces when appropriate.

When I get the sheet metal cut/bent ordered I’ll post here to give everyone an idea of what it would cost, or atleast the floor of what a build could be.

3 Likes
56

I was planning on making my hdd case similar to the 3D printed video I linked earlier, except out wood for the box, and drilled aluminum for the hard drive mounting bars that I can get in 4ft lengths at Home Depot.

57

It took me a while to find it, but if your primary objection to disk shelves is lack of filter, there are several products that completely enclose the pc in dust filter fabric.

A friend of mine had his computer is a pottery studio, and used something like this for the inner layer Protecting his computer from dust.

Also due to it’s large surface area there is little pressure drop.

2 Likes
58

By the way, using little bumpers to dampen vibrations works really well. Before there were commercial products, the two most popular options were neoprene pads, and hearing protection plugs, which often came in 50 packs.

59

Current progress:

image
image1920×1672 125 KB

image
image1920×1337 168 KB

image
image1830×1879 271 KB

I decided against giving the HDDs their own individual access door, it would add to much bulk to the case and make it weak; they will be accessed by sliding the case apart which should work better now that I added cots rails for it.

I still need to model the back tie plate for the case. I’m not 100% sold on putting the psu right above the motherboard because it will restrict cooler height, but remote mounting it in the middle of the case is probably a can of worms with cooling and the added complexity.

2 Likes
60

Hi Anton,

Pity the D8000 is discontinued I use one as well for the same reason, great case

1 Like