I’ve tried a few different drives in my server over the years.
Rarely directly as NAS drives though. It seems a little wasteful over the network, as you are never going to utilize that level of speed even with very fast enterprise networking, unless you are hammering the NAS box with a large number of clients in a big office.
Single client network storage speeds start scaling very poorly somewhere above 10gbit currently. Even when running data directly from RAM on the server, I can rarely achieve speeds greater than ~16Gbit/s, even on my 40Gbit NIC’s.
My main NAS does have some NVMe drives in it, but only in “supporting roles”. I mainly use ZFS on the NAS side due to its superior data protection, and while I have tested some all NVMe pools, they have never lived up to expectations, so I have never wound up actually putting them into service.
The supporting roles I am talking about are as various types of cache and other types of implementations that accelerate the main pool.
I currently have 12x 16TB Seagate Exos x18 drives as the main pool, configured in two Raidz2 VDEV’s with six drives in each. (This winds up being the equivalent of hardware RAID 60)
Then various NVMe drives help out in other ways:
-
SLOG drives: This is a ZFS specific type of device that speeds up sync drives. It is tempting to think of them as write cache, but that’s not really how they work at all. The spoecifics are beyond the scope of this thread, but suffice it to say, if your workload requires sync writes (in other words, the writing program needs confirmation that data has been comitted to NVRAM before sending more data) these drives help speed up those writes significantly.
-
L2Arc or Cache drives: These work as read cache for data on the slow hard drives.
-
“Special” Drives: These are another ZFS-specific way of using drives. The “Special” devices are used to store metadata for the pool, speeding it up significantly, so the slow spinning drives and their millisecond seek times don’t need to do this work. They can also be used to store small files and speed up their access.
As for which drives are good? Well that depends.
You will find many people who will absolutely fume at the concept of ever using consumer drives in a server or NAS. Their perspective is that if it isn’t an enterprise or server drive it doesn’t belong. Proper enterprise drives can be a little pricy for us enthusiasts though. At least unless you are willing to buy older, slower previous gen drives from server pulls.
Here are some drives I have used though, and I’ll give you my experiences with them.
1.) Optane drives: These are going to be small, and expensive, and since Intel discontinued them years ago now, harder and harder to find. They are great if you can get them though. Their extremely low latency makes them suitable for the above mentioned SLOG drives. (Also referred to as dedicated ZIL drives) I am a firm believer that there is no better drive, and that there probably won’t be for a very very long time, but the small size, expense and difficulty in finding them at reasonable prices now may get in the way of actually using them.
2.) Consumer Samsung: While I had great luck with Samsung Pro drives back in the SATA SSD days, in the NVMe era their drives (even the Pro ones) are just not very suitable for server use. While they work great in desktops, I have had them randomly drop out and require power cycles to come back when used in servers that have long up-times. I’d recommend you steer clear for this application. Note, Samsung does make some actual enterprise drives, (models starting with PM, like PM983, etc. I have no doubt these are decent server drives, but I have never used them myself.)
3.) WD Black Drives A couple of years ago I found a shockingly good sale on 4TB WD Black SN850x drives, so I bought two. I have been using these two as cache drives for my main storage pool, and they have been excellent. They have worked every bit as well as I would have expected from real server drives. They ahve been bulletproof from a reliability perspective, fast, and write endurance has been shockingly good considering they are TLC drives. I don’t know what the current status of these drives is what with WD and Sandisk splitting up, and WD focusing on hard drives again, but if you can find them, I think they are a good option here.
4.) Inland Premium: Inland is Microcenter’s in-house brand. Their “Premium” drives were surprisingly good for a discount store brand. I think I found some data that they were made by Team for MicroCenter at some point, but I am not 100% sure. I bought a huge pile of these and have used them all over the place. They have been really reliable in a server application. The larger drives have had good write endurance too, but smaller models (256GB) have worn faster, as would be expected. I would caution that you not project my success with Inland Premium drives to other Inland branded drives. They are a retail house brand of relabeled products from other manufacturers, and thus one SKU of Inland drives will vary greatly from another.
So, Enterprise drives are going to be pricier but more reliable. Consumer drives are going to be hit or miss. Some might be excellent, but others might really struggle in a server application. So, if you go this route like I have, make sure you have both good redundancy and backups!
Worth noting though, you don’t have to bother getting expensive latest gen drives if all they are going to do is serve data over the network. Even the oldest Gen3 drives will by far saturate a 2.5, 5 or even 10Gbit network interface, and over 10Gbit - as previously mentioned - network storage doesn’t scale very well for small client counts. They work better at handling combined requests from many clients.
Heck, I’m not even sure it is worth bothering with NVMe drives at all. A well constructed array with SATA (or SAS) SSD’s ought to easily be able to saturate a 2.5gbit link, and may save you a lot of money.