I’m looking for a RAID controller that does hardware RAID for PCIe3 X8, for which there are plenty, that is NOT an enterprise level card with an expensive price tag. So, looking at the $150 - $300 range, MAYBE a little more.
The requirement for RAID is pretty simple, but would prefer that it could configure 2 separate arrays, as I need two RAID 0 configs, and with the amount of work I put them through, and I have an 8 core CPU that at the same time I’m browsing the web and doing other work means that a software RAID is not ideal.
I want one that works with available drivers so that the config for it under Linux is easy. I’m not loading an OS to it. I use Ubuntu Mate. I know the DE makes no difference, but maybe the fact that it’s Ubuntu can? I’ve seen drivers listed for different variants of linux. If you know of a particular model that works with the latest versions of Linux I would greatly appreciate the info.
Please, not looking to debate the merits of different types of RAID. I need the write speed, and I can’t add additional drives to the system for any other form of RAID, nor do I want to because I back up ALL my data to external drives anyway, which I can do at my convenience. It works great, going on 15 years now, but in the past I was always able to get away with the MB hardware assisted software RAID, or FAKE RAID as some people call it. I can’t do that now with my current system, and my demands are heavier anyway.
Have you looked at using mdadm software raid? The reason why I list this is it is used in quite a few applications like external NAS devices provided by brands such as QNAP, Synology, & Seagate.
Since you are on Linux, you should already have that software ready to be used.
I asked someone who developed drivers for these things how that works recently. He told me that really all the “hardware” (possibly actually a firmware module) does is basic support for booting from such arrays, everything else is in software.
Yeah I thought about it plenty, so much so that I know I need a hardware RAID, not a software RAID.
The point is, I use the PC for multiple things at the same time and I don’t want to lose responsiveness because one or two cores are tied up with RAID. It is honestly getting pounded, to the point that EVEN WITH hardware assisted software RAID, with what I do, the work slows down and my memory of 32GB can get full because I allow applications to buffer to allow time for the drives to write. I want my drives to keep up AND cores not to be tied up.
I also can’t have this being a NAS, where better quality ones have their own controller. I don’t have space for another unit. So, must be internal to my case that holds 9 drives.
Well, yes and no. Cheaper RAID controllers are basically the same thing that comes on a MB. The correct term for this is Hardware assisted Software RAID. They have the benefit of adding ports along with the control. That is what you just defined. These are controllers that tend to cost little money, because like you said they have firmware that allows for the definition of the RAID and that’s about it. It will also typically allow an interruption of the boot process so it’s information can be read and you can boot to the drives on that controller.
This is in no way Hardware RAID. Hardware RAID has the firmware chip for the system to read, along with an accelerator, along with RAM, along with a mini-processor to handle ALL the RAID functionality. These are more expensive controllers. Many of the SAS/SATA PCIe3 X8 RAID controllers are true hardware RAID devices. I’m looking for those who use these and can recommend a particular model or brand that has up to date Linux drivers, which is typically where the problem lies.
This is really a specialty situation because normally this setup ends up in servers. I don’t have space for any other units, and my 2 machines have cases that hold lots of drives and I have to make the best of them. I bought the cases I have specifically for this problem a few years ago, and if I were using Windows this wouldn’t even be a question because there are PLENTY of controllers that have up-to-date Win10 drivers.
The difference between Enterprise versions and lighter load server versions is the price tag and the amount of drives they can control, where with Enterprise variants, they give external SAS connections to connect to a drive bay, and can control more drives than what a typical business would need connected. The limitation ends up being the fact that it’s a PCIe3 X8 controller, so the bandwidth of the controller gets limited to approx. 8GB/s, which is WAY beyond what I need.
Right I quoted “MB hardware assisted software RAID” because that’s what I was talking about. I know I didn’t answer your question but it seemed like a somewhat interesting anecdote to share.
As for your interest in a cheap RAID controller, I encourage you to measure the performance of a software RAID solution in Linux. CPUs are incredibly more powerful than a RAID controller, there is really not that much overhead for a modern processor. If you measure a bottleneck and it is indeed CPU bound, any fairly recent MegaRAID controller can be obtained within your budget second hand due to being extremely widely used in servers. I have seen older mfi devices fail to show up in the Fedora CoreOS preview installer but that’s been the only fluke in my experience.
I don’t want a cheap RAID controller, but I guess that’s relative to what a person makes for income. I gave a price of $150 - $300 which pretty much eliminates the Hardware assisted Software controllers, which tend to be less than $100, and puts you in the realm of true hardware RAID controllers, which takes ALL the work off your CPU.
I’ve looked at the MegaRAID line, and the choices are many so didn’t know where to even start, so I guess I’ll contact the company to see what they say. MegaRAID controllers can also be bought brand new, and within my budget, as long as they have updated Linux drivers that will support Ubuntu., which is really the summation of my question. Which controllers actually have updated Linux drivers that will work with Ubuntu.
A CPU makes a GREAT RAID controller, unless your CPU usage is already pretty high, then the fact that it’s controlling a RAID becomes a drag on your system. This was already happening due to heavier loads and why I know I need to completely offload the RAID functionality.
Yes, this is the category I’m looking for, but unfortunately without contacting Broadcom I don’t know for sure about the drivers, because Ubuntu is listed, but only up to 15.04, and currently Ubuntu is up to 19.10
This one is also a TOUCH above my price range. Hopefully enough fishing will pull out a person who’s using either the Broadcomm (LSI) or Highpoint or Adaptec or SuperMicro devices. Those companies all have PCIe3 X8 controllers, but there are literally 50 - 100 of these to wade through and a LOT of searching through specs and typically finding a lack of info about updated Linux drivers, for which is that doesn’t exist, it doesn’t help. I doubt that one exists that will set up 2 separate arrays. I’m thinking that’s a dream that won’t come true.
Thanks, and yes I’ve already been on their pages looking at different models.
Linux has the driver in tree, mpt3sas. As for the price, like I said, it’s not hard to get it second hand for a price easily within or below your budget. Setting up multiple arrays on the same card is a pretty standard feature.
They’re all over ebay. Not really any moving parts to wear out on these, shouldn’t be any trouble buying used except maybe battery life if you’re looking at one with a bbu.
How do you get to that? Even if I had seen this site before, when I search for SAS 9270-8i, it won’t pull up anything even if I drop out part of the characters I don’t get to that driver.
Hi,
The cards reccomended by Freqlabs look like a good option.
Personally I went with cards that are a bit older, because they typically have wider support, with more drivers available.
I have gone with several PCIe2x8 cards, providing 8 SAS lanes/2ports each.
The types of drives you have might be taken into account; SSD’s might maybe be bottlenecked, but only on sequential thruoughput. the Random read / seek / iops would still be good.
SAS/SATA would still be fine on v2x8 cards.
Also the motherboard needs to have enough lanes, as it might already be throttling SSD’s on the built in SATA ports (Thanks DMI) but I didn’t read the whole thing properly.
TLDR: wand hardware raid, for only a couple hundred bucks.
Desire PCIE3
Might have to settle for PCIe2, and save ~$150 using second hand card from ebay.
Whichever way you go, If using Sata drives, check the model supports SATA natively, as you’ll be using the build in card’s bios. Also ensure you get the correct /long enough cables
Im not in the loop when it comes too professional storage. Is there any companies making PCI4.0 cards yet ? Its got to be a thing in the next 2 years untill intel makes PCI5.0 a thing not vaporware.
I can’t drop down to PCIe2. I guess I could, but I’d be depressed. One of the drive ops is similar to pounding a drive with continual random writes for hours on end, with files in the size range of 256KB to 8MB. So continuous write with large files never comes into play. This is more like a worse case scenario for drives. Samsung SSD 860 EVO 2TB is about as good as it gets on my budget, and MAYBE I could split the work between 2 non-raided SSDs of this category. On the other hand I need one large RAID, and I have 2 Seagate EXOS 4TB drives for that. I also have one 1TB NVMe, a Sabrent Rocket. I can probably get away with a software RAID on my other system. But yeah, I need the RAID controller for SURE with the mechanical drives. The speed of the SSDs could possibly keep me out of a RAID config, but then I get into more moving of files. I think that their worst case write (nonstop random with files under 1MB) will handle the speed without RAID.
Not that I know of. I don’t know if there is a hurry to make them, if they ever will be. The issue is the possible pass by of PCIe4 by Intel, Nvidia, and then consequently the MBs, to PCIe 5. This has already been discussed, and not just rumor.
I’m dubious, because it’s such a large jump over PCIe3, and the switching circuits are going to have to clock WAY faster, and cost more, and to me trying to incorporate PCIe5 sounds like an expensive task that isn’t necessary for another 10 years.
But that’s the issue. The X570 boards will be fine one way or another, because PCIe5 has to be backward compatible with PCIe4, or you would get lawsuits flying, as it’s in the spec for PCIe5.
Oh, and if Intel, Nvidia, etc… make the jump to PCIe5, AMD will also with Zen4.
You have an 8 core CPU. You’re not going to even notice RAID operations happening especially if you’re using RAID-0, RAID-1 or RAID-10. There are no complex calculations happening in those.
The hardware limitations are almost entirely in the PCIe lanes allocated to the on-board SATA controllers. If you’ve got a stack of SATA SSDs connected to a motherboard you can overload the board’s PCIe bandwidth.
But the actual disk operations are happening via DMA through the SATA controllers and won’t bother your CPU at all.
Now, if you were doing RAID-5 or RAID-6 onto SAS SSD drives then yeah you’d want a controller card and a battery backup for it.
But I don’t believe you when you claim software RAID will slow you down. People run dual M.2 NVMe drives in Linux at 4 GB/s in software RAID-0.
Anyway, I’ve got a Ryzen 1700X running on an Asus x370-Pro motherboard with 8 SATA drives on the board controllers, using BTRFS RAID-10. It runs at full speed of the hard drives. Since they’re mechanical spinning WD Reds, that’s not too demanding.