I am looking to buy 3 x 8TB SSDs for a RAID5 setup for my home PC. For some reason I couldn’t find much options. One is the Samsung 870 QVO which is 3+ years old tech, and the other one I found is Western Digital WD Gold Enterprise NVMe that comes in U.2 format and seems unreasonably expensive for my purpose.
Main reason I want SSDs is because I don’t like the noise from the HDDs. Buying the 870 feels like investing in old tech. Any advice?
SSD prices have fallen in the last year or so, but only for the “small” capacity drives (up to 2 TB). Regrettably, 4TB and larger drives are more difficult to make so their prices stay high. This isn’t helped by HDD manufacturers making a much, much bigger profit on their equivalent high capacity HDD’s and if you buy their SSD instead, they still want that big margin so you’re paying through the nose 
I recommend getting used enterprise drives. SAS drives are easy to connect in droves, can be extremely cheap per capacity at times, and should be relatively reasonable to cool. NVME U.2 drives may need some more exotic cooling, and can chug quite a bit of power, but are more broadly available, and will likely saturate the market in the coming years.
High capacity SATA consumer SSDs are basically a dead duck, and the few that do exist are trash tier garbage quality suicide drives that nuke themselves from the inside out.
I’ve seen auctions for 15TB SAS SSDs go for only a few hundred per, but I suspect bidding on those, they would quickly rise to around 600~700 a piece, which is about general NAND prices.
There’s also the option to get 2~4TB drives, and just plug in more of them. 3x drives for Raid5 isn’t really optimal anyway, especially since good SSDs have such a low likelihood of failure on reads. I wouldn’t trust SanDisk or TeamGroup or the like, or pretty much any consumer dramless SATA drive for that matter, but enterprise Micron or Samsung or the like, you can pretty safely run a much nicer 5:1 or 7:1 ratio without too much risk.
I recently decided to go all flash on my home NAS for … reasons. Other than the Prime Day sale for the trash-tier Samsung QVO drives, at 4tb+ the best $/tb I saw was for used Gen 3 U.2 drives. I picked up some Kingston DC1500M 7.68TB drives for $320 each. Someone posted on STH and reddit an ebay seller in the UK with some used 15.36 TB NVME for $500 + shipping from the UK. As more Gen 3 NVME moves out of the datacenter, I’d expect prices for U.2 drives to keep falling. Of course they use more power than SATA/SAS/M.2, and you need gobs of PCIE, so there’s that.
RUN! Run and don’t look back. They are really, really, really bad.
True that.
At 8TB, they probably work fine as WORM / low write devices.
Smaller drives would have churn, and rewrites, but large? It’s Fine
What models would you recommend? Are Gen3 U.2 drives the same as NVME U.2 drives? Also are they generally better than SAS SSDs?
Can you elaborate please? Doesn’t low failure rate mean you need less redundancy, i.e. less disks, not more? What setup would you recommend? If I will be getting a lot of drives, I might consider going the NAS route. My main concerns with that are second chassis mean more fans and more noise and also don’t have experience with network access speeds and whether network cables will bottleneck performance (I hate wifi, so only cables at home).
Damn, these costs $1700 per, new, where I live (that might be very old price though, there’s low demand here). Aren’t used drives pretty beat up though? Especially data center drives?
Although, I am not planning on getting any, can you say more about why these suck? I am just curious.
Gen3 U.2 drives the same as NVME U.2 drives?
There is no difference, gen3 in this context means PCIE gen 3 nvme drive with u.2 form factor and electrical interface.
Good rule of thumb is the following:
Although, I am not planning on getting any, can you say more about why these suck? I am just curious.
These are sata QLC flash drives, so once you run out of write cache, performance tanks so bad it unbelievable. Do you want ssd drive that can perform worse than spinning rust in sequential writes? With QLC you can !
They might be cheap, but they a re not cheap enough given the final performance.
They might have their uses, but you really need to know what you are doing, else you will be very disappointed in the end
Behold the trash tier SSD performance 
Different drives for different lives. U.2 enterprise drives can have very high power(25w) draw relative to equivalent SAS(10~15w), which is often very high compared to SATA(3~6w). It’s not a rule, though. NVME drives can also be 5~10w or less, or have some degree of background/idle states, as can SAS drives. It just depends what the drive is made for.
Enterprise drives also often don’t have idle states, because if your server isn’t actively doing something all the time, it’s wasting money.
U.2 is just a 2.5" form factor for PCI-Express, or Non-Volatile-Memory-Express.(NVME) Strictly speaking, it’s faster than SAS, but what’s inside the drive is what matters most. You can sometimes find the whitepaper or specifications for the drive in question online.
For U.2, not only is there power consumption to consider, but also cooling. Simply pointing a fan at the drive may not be enough on it’s own, as they’re designed for streamlined high-airflow environments. 25w in a 2.5" box doesn’t cool very easily, and can end up overheating or thermal throttling, depending on the drive design.
I don’t have a lot of first-hand experience with a wide array of U.2 drives or anything, just a couple NVME M.2 enterprise drives(one overheats, one throttles, both required some extra heatsinks to behave well, neither has any kind of idle state) and second-hand experience from someone who purchased a cheap 7.68tb U.2 Samsung PM1723A and needed some extra heatsink to keep it cool.
Low failure rates means you need less redundancy, but redundancy will always need to be at least one disk, so more disks = fewer redundant disks and more space.
3x drives, in order to have redundancy, means that 33.33333% of your space is spent on parity data. With good quality enterprise SSDs in a home environment, it’s relatively safe-ish to run more drives with fewer parity drives, since the odds of back to back drive failure during resilvering is very low.
A 7:1 ratio means 12.5% loss of raw capacity for a single drive failure level of redundancy. With good quality enterprise drives in a consumer workload, this is probably fine.
RAID is not a backup, so do keep backups of everything important, but you’re unlikely to be set back more than the resilver time if you do eventually lose a drive. If you lose two drives, though, you do lose everything.
I don’t really have any specific recommendations for drives to buy, but I do have some general rules for drives to avoid.
If it’s SATA, it needs a dram cache, period. Anything using SiliconMotion controller is suspect. If you can’t find information on what’s in it, and it’s a consumer-facing brand new drive from amazon or newegg, assume the worst until proven otherwise. If it’s QLC, don’t count on it to be reliable storage.
For NVME/M.2, if it doesn’t have a dram cache, make sure it does use Host-Memory Buffer, or HMB. If it doesn’t, it’s just as bad as the SATA drives.
Look for drives with detailed spec cheets or whitepapers. If endurance is rated in DWPD, multiply this by the number of years warranty * 365 for a broad idea of how many writes it can sustain in a fairly optimal environment. Compare this with the amount of writes already on it before purchase if you can, though the majority of enterprise drives flooding the market right now are from failed businesses or drives written off as EOL based on warranty, rather than drives nearing the end of their write-life. If someone is selling bulk lots of 8+ enterprise/prosumer drives, it might be worth the gamble.
As a general rule, consumer facing drives are designed to fail more quickly, in order to maintain a certain degree of new-in-box drive purchases. Things from Sandisk, TeamGroup, Patriot, ADATA, Lexar, Samsung Qvo or newer Evo drives, all these things should be treated as suspect, and many more as well.
Older Samsung Pro SATA drives can be quite good, though, if you can find them. Older WD Blues and Reds are fine, too.
To add a bit to what greatnull has said, the low-end of enterprise drives can actually be pretty low-end. SanDisk Skyhawk U.2 drives can be rated for as few as 0.5DWPD over 5 years. I believe there are some drives rated for even less than this, but I can’t remember what and might just be crazy.
For consumer workrates, though, even that’s still pretty good compared to what you’re likely to get out of a dramless SATA SSD, much less a QLC one.
Noticed that Samsung has a 4TB Consumer NVMe 980 Pro. This might be a dumb question, but can’t Samsung just stick 2 of those in an SATA-like SSD enclosure and turn it into an 8TB SSD offering somehow?
It wouldn’t make any sense economy and complexity wise. Such design would also be nighmare from reliability and support side.
It cheaper and easier to simply put more nand dies to existing m.2 sized pcb, but where isn’t market, there isn’t product.
Samsung can certainly put 30.72TB or more in a 2.5" form factor. The question isn’t “can they do it” so much as “why would they want to sell it to you”.
You can buy a used one, if you have a few grand to drop. If you want one new, call up samsung and ask for a few hundred of them? Maybe you can trade a few acres of land or some expensive cars for it or something.
The reason 8TB QVO drives suck isn’t because they can’t turn a profit selling you a good fast 8TB SATA SSD, but because then can con you into buying a garbage tier 8TB SATA SSD today, and an even more garbage tier 16TB SATA SSD tomorrow when the 8TB one dies prematurely.
100TB+ 3.5" SSDs have existed for quite a few years now. Sadly, I do not have enough internal organs to afford one, so I’ll stick to the 2TB enterprise scraps like a good peasant.
Noticed some relatively new 8TB NVMe popping up on the market, most notably:
Curious what you guys think about those products as they seem to fit my goal at first glance.
at least on paper, they look very good. 176l Micron flash, Phison controller, 2gb onboard cache, 2.4% overprovisioned.
unless they’re configured poorly, they should actually be quite high end and worth using drives.
Of course, they’re no doubt very expensive, so it may be worth looking at enterprise scraps if you’re looking at prosumer drives that re that high end…
Do you think these can be used safely as home NAS drives so I can finally replace my 12TB noisy WD Red? Or are consumer NVMe write/reads still very limited (in terms of total lifespan) and should be used for less intensive stuff only? Not that NAS is very intensive, but still there are days when there is 24/7 writing/reading. thanks.
These look fine. I’d watch temps for a while just in case, but I don’t think you’re going to eat 6PB very quickly. The NAND flash it’s self will be a non-issue, and I don’t think these higher end Phison controllers will try to eat it alive if you have dram cache available.
With that said… How expensive is it, compared to something like these?
You just have to really watch out for those low-end consumer SATA SSDs, because they’re made to suicide through poor design choices, like having no dram cache on a sata SSD.
If it has a dram cache, or is nvme, and it’s 3-bit per cell flash or better, it’s probably safe.
I find it difficult to navigate enterprise products, because it always turns out there’s something odd about them that you wouldn’t care about if you needed it for a data center, but would care a lot if you need it at home. Like some weird power connector, or extremely noisy, or needing some special RAID card to get it to work etc. Other than that they probably make a lot of sense if you know what you are doing.
Well, the good thing about sas is, the odd thing about them you wouldn’t care about in a data center but would at home is, you plug them into a SAS HBA, which are cheap and available all over ebay. That, and they typically draw a bit more power than a SATA drive would, but it’s not far off from consumer NVME drive power draw. I actually forget which is higher.
The thing to look out for is which SAS connector the card has, and that it’s running in IT Mode
The product name will have something like 4i or 8i in it, indicating the number of drives you can connect. It will also have 1 SAS connector per 4 drives you can connect; this is normal and what you want. If you see a bunch of SATA connectors instead, these are still SAS cards, but avoid them anyway because you’ll need to buy more junk to use them.
LSI 9207-8i 6Gbs SAS 2308 FW:P20 PCI-E 3.0 HBA IT Mode For ZFS FreeNAS unRAID | eBay
This card is 8087, as are most of the cheaper SAS cards.
Look at the shape of the connector, and copy the name; it’ll be something like SFF-8087 or something. Other than the card, you need the SAS breakout cable to match. It should look like it fits the plug on your card on one end, and plugs into 4 very wide SATA connectors on the other. This is because these connect to both power and data at the same time. Aside from that, you plug standard SATA power into each of the cables.
Mini SAS SFF-8087 To 4 SAS SFF-8482 Hard Drive Forward Breakout Cable RAID HBA | eBay
The cable costs as much as the card most of the time. Note the wide connection with a plug for SATA power on the back. That’s what you need to plug into a SAS drive. The cables with smaller SATA datas on the end will need an adapter, and you cannot plug SAS drives into a SATA port, even with a SATA → SAS adapter; those are for plugging SAS drives into a SATA cable connected to a SAS port.
And that’s it! It’s just SATA drives with extra steps. You might need to format from 520 to 512, but that’s just typing some stuff into a terminal, and there’s a topic on the level 1 forums tutorialising that process with very clear instructions for SAS SSDs.
But, it’s up to you if the price premium is worth it to not have to think about plugs and cards, or wait for a used LSI card and expensive cable off ebay that’s shipping all the way from china. There’s nothing hard about using SAS SSDs, like there is with U.2, where you have to worry about actually cooling 25w of SSD, or having enough PCIE connectivity to plug them in anywhere. But, depending on the money you have to burn, and how much you value your time/are scared of PCIE cards, it’s a valid choice either way. I don’t judge. 
I use Intel P4510 8TB, they’re U.2 so you’ll need to be able to bifurcate your slots or use a PLX card and hang them off that.
If you haven’t used U.2 before just understand that they suck quite a bit of power (up to 20watts) and as such make a good amount of heat so airflow is a must.
