SSD for all ssd Freenas pool

Hardware
, ,
1

Howdy :slight_smile:

I’m looking into making an all ssd pool on my freenas server, to improve responsiveness to small file access.
My current raidz2 10x4 TB HDD pool is filling up, reducing performance. So I wan’t to move pictures, documents, music and other small files types I often access to all ssd.

At the moment I have 2 sata ports left, so I’m thinking a mirrored vdev of 2x2 TB ssds.

I won’t be able to use the full bandwidth they offer, since my network is 1gig, and I can’t add a 10gig nic, since I had a brainfart when I built it, and chose mini-itx… The pci-e slot is occupied by a hba card.

(I do plan to build a new one within the next 6 months, when Truenas 12 is released and the new Supermicro H12 Epyc motherboards are released. With 10 gig networking and all that good stuff :slight_smile:)

But I’m not sure which ssd to chose for an all ssd pool.
I’ve been googling and going trough old posts here, on Reddit, ix systems forums and on servethehome, but the only time they talk ssd is for l2arc recommendations.
And the threads I’ve found on all ssd are 2-6 years old, so nothing recent.

It’s not gonna get hammered with frequent rewrites of data. Ones it’s on there it’s pretty much static. Files only get added over time, not changed much after it’s been added.

I’ve been looking at the following ssds:
Model name - Price - Write Endurance - warranty

  • Kingston SSDNow A400 - 193 Euros - 600 TBW - 3 years
  • Crucial MX500 - 228 Euros - 700 TBW - 5 years
  • Samsung 870 QVO - 282 Euros - 720 TBW - 3 years
  • Crucial Micron 5210 ION - 298 Euros - Unknown TBW - 5 years
  • WD Red SA500 - 300 Euros - 1300 TBW - 5 years
  • Seagate Barracuda 120 - 301 Euros - 1170 TBW - 5 years
  • Samsung 860 EVO - 313 Euros - 1200 TBW - 5 years
  • Kingston Data Centre DC450R - 326 Euros - 1301 TBW - 5 years

The Kingston A400 & Samsung 870 Qvo only has 3 year warranty, the rest is 5 years.

The Micron 5210 ION and Kingston DC450R are enterprise SSD’s, so they shoulde hand being in an array pretty well, their write iops are lower than the consumer ssds. But still better way than HDDs. My guess is they don’t have all that turbo/slc caching the consumer ssds have. Probably to make the performance more predictable.

The Samsung 870 QVO and Micron 5210 ION are both QLC nand, but since I’m not hammering my nas with constant rewrites or runiing vm’s from it, I don’t think it’s an issue.

The price difference from cheapest to most expensive is over a 100 Euros, and the endurance is double.

From the reviews I’ve been reading about the WD RED SA 500, it’s basically a WD Blue SSD with tweaked firmware, but the 2 TB blue has an endurance is 500 TBW, and I don’t quite get how the doubled the endurance with a different firmware?

I haven’t included any SSD’s I cant afford, but I’m not looking to spend more money than necessary if it doesn’t benefit my use case. It’s not write intensive, so I don’t think the +1000 TBW of the more expensive ones will make a difference for me.

Input is welcome ::slight_smile:

2

The mx500 look the best.
Not much chance of hitting the TBW, but using it as a confidence metric by the vendor, means they trust the chips to last. It’s also one of the more tried models listed, and still going strong.
Even though few writes or P/E cycles, I would still avoid QLC until they are like half or two thirds the price of TLC, but they are not that different yet, in my opinion.

I have a couple A400’s, and they work fine for me, just not the fastest.

1 Like
3

This is what I use in my all SSD ZFS pool. They are mirrored with a hot spare currently. Performance is top-notch with over 11 virtual machines running on them.

1 Like
4

With regards to the WD and Seagate drives, I haven’t seen anything apart from their brand names that even come close explaining the premium price for average drives.

I personally am looking at used Samsung pro drives, and hoping there are some deals. The EVO’s are good, but I don’t thing quite good enough for cheap NVMe prices. Not that you have m.2 sockets available

1 Like
5

Yeah, it looks like the MX 500 is the perfect middle ground. TLC, plenty of endurance, good performnce and not to pricey. And a 5 year warranty.

It is kinda weird how qlc is supposed to be the cheaper option, yet there are plenty of tlc ssds with just as good performance and endurance for cheaper.
But it’s propably a matter of time, as production ramps up and they improve the manufacturing process.

Good to hear :slight_smile:
I won’t be having a hot spare, since I’m out of ports, but all data are backed up after the 3-2-1 scheme.

I think I’m gonna go for the MX500 :slight_smile:

6

Not sure what your use case is, but for me I’ve had mine since April 13th and since then I have written 7.14 TiB of data to them.

The 2TiB MX500 are rated at a TBW of ~700 TiB, meaning that in ~4 months I’ve used only 1.53% of my drives write endurance.

According to my calculations I write an average of 1.785 TiB per month which means these drives should last me roughly 32.6 years based off my current rate (if we look at only the TBW).

TLDR: rated for 140 TiB/year of writes.

3 Likes
7

MX Ones are the only one that guarantee cache write-through with power loss and they have the CAPs to take it.

Regarding the longevity - if you avoid data that often changes individual bits you are golden. The nature of SSD is that you need 4kB to rewrite a 512B sector. This means that saving changes often will reduce the longevity quite a bit.

Now that is easily solved by using smarter DB like software.
If you are storing normal files, just go with the best for the buck.

The whole idea is kind of weird as the main limitations of NAS performance are usually not limited by the storage options. The real downside to NAS performance are inconsistent predictions.

Not all use cases will benefit from SSD - for example mail cache for SOHO will benefit greatly, while browsing photos will suck on whatever.

That said if you have off-server backups SSD are perfect and there is little to no need for spinning rust on your main NAS

2 Likes
8

I presume it also helps in some ways having a COW file system like ZFS, as it does not re-write blocks/pages too much

But it hinders Trim / wear levelling because the drive can’t always shuffle stuff around so much.

Trim is being worked on, and should be fixed before too long.

Oh, and I did not think/realise MX drives had any power loss protection; I presumed they would be like a normal consumer drive.

2 Likes
9

TRIM shipped in ZoL 0.8.3 I think. Which is the latest stable.

TIL too. Though, my server has a writecache battery so I’m covered anyway :stuck_out_tongue:

10

Storing +300.000 photos, documents, music, digital mangas. All the small files I randomly access, which I’m starting to get sluggish access time to when browsing. My current 10x4tb raidz2 array is 70% full, and the small files are scattered across it. I’m nearing the 80% max rule of thumb.

How do you check how much they have written?
Is there a cli command to check, or is it from smart data you see it?

Sounds like plenty :slight_smile:

Neat, I hadn’t seen that. That’s just gravy :slight_smile:

It’s static storage of +300.000 photos, documents, music, etc. Pretty much write once read many.
Only documents change once they are added.

Backed up to local backup unraid server + Google drive via encrypted duplicity :slight_smile:

I’ve also read this a few places.
It might be included with Truenas 12 and ZFS 2.0?

Mine’s hooked up to a ups, so I’m not too worried about a sudden power loss during writing :slight_smile:

1 Like
11

Its in the SMART data.

Total_LBAs_written

Good. But yeah mine is overkill. UPS. Cache battery. Redundant PSU. God I love enterprise shit.

1 Like
12

Neat. I’m gonna check my boot ssds when I get home from work. I don’t expect them to have written much. If anything :slight_smile:

As long as it can be in a separate room. My computers are in my living room. I don’t I would enjoy a screaming rack whilst watching a movie or eating dinner :slight_smile:

13

Oh, and I did not think/realise MX drives had any power loss protection;

All drives do, just the cost savings and focus is usually elsewhere. Modern SSDs can be blamed for a lot of possible issues and manufactures have priorities. 4GB DRAM cache write can take even minutes and drive has to be online to finish it.
This is important to pre-SSD notebooks where voltage demands of an SSD can actually trigger FW protection and prevent notebook from recognizing the drive.

But it hinders Trim / wear levelling because the drive can’t always shuffle stuff around so much.

Trim is only important if you shuffle data on and off. Benefits to longevity are then questionable. It is more important to performance.

For normal documents TRIM is barely needed. For doing incremental binary backups from your desktops TRIM becomes a must.

14

I only have one mx500, and a bit back I starting getting error notifications but the drive was fine. Others had the issue pop up as well.

I’d thought I’d pass along the solution for anyone else that has one, it wasn’t a drive, instead it was smartmontools giving a false positive.

https://www.smartmontools.org/ticket/1227

2 Likes
15

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.