I am currently running a pretty big server mainly focussed around media streaming through Plex. I’ve run this for 2 years of a Dell T420 with 2x e5-2470V2’s. 20 cores all mainly dedicated for transcoding together with 96GB RAM.
I’ve always run my main storage of an Dell H710 with 8x WD RED 4TB disks, which are running in RAID5.
Since I started reading into the discussions of RAID and ZFS, I also read into the low recovery rate of such a size pool using hardware RAID. Therefor I’ve bough a Dell H310, flashed LSI IT firmware on it, so I can run the card in HBA mode. This weekend I can lend some hard drives from a friend of mine, to temporarilly transfer data to these drives so I can convert my pool from hardware RAID to a ZFS pool.
Because I can’t really chance much after I’ve setup the pool. I’m asking on this forum what setup/settings I can use best. I’d preferably run RAIDZ for maximum storage, although if this means my data is far from safe. I am willing to go 2 disks parity. I do have one hot-spare 4TB disk at my disposal.
I’d like to know the views you guys have on ZFS and what settings to use. Thanks!
I’m new to ZFS (on Linux) and have a pair of NVMEs set up in mirror to store virtual machines. So far, so good, but I also have not experienced a crisis that would put it to the test. But it compares favorably to my experiences with RAID1, which were a total disaster.
For my media server I just do unRAID, that has been solid for me for several years now.
Thanks for the reply Psydork, however RAID1 cannot really be compared to this usecase. If a disk in RAID5 fails and you install a new disk in order to rebuild the array, there is a pretty substantial chance another might fail during the stresses of rebuilding an array. I’m wondering wether to run RAIDZ or RAIDZ2 and what settings to use for maximum performance, I’ve got plenty of RAM that can be used as cache.
I am curious though how a RAID1 mirror would fail on you. I’m running Proxmox VE as my hypervisor
The raid mirror failed do to corruption of some sort, which I guess go mirrored itself. On two occasions with two different drives, I had system crashes and rebooted to find the partition tables gone. Anyway I was just commenting to give my general endorsement to ZFS. I found this helpful: https://www.svennd.be/basic-zfs-tune-tips/
I’ve been running a RAIDZ2 (2 parity drives) on my FreeNAS ZFS system for a good 5 years at this point. I started off with 8 5TB drives back in 2014, I’ve only lost one drive in that time and the RAIDZ2 had no issue swapping out the drive and rebuilding. I just recently even upgraded the entire system, slowly swapping out all 8 5TB drives for 10TB drives, again I had no issue (though rebuild times per drive took ~6-10 hours). Early on I was only running 16Gb of RAM and I had some pretty major latency issues due to the cache filling up so quickly. I upgraded to 32Gb of RAM pretty soon after that and have never had a latency issue since. Even with upping my total storage from 24TB to 50+.
Both links you posted are very contradictory. mirror vdev’s are way too inefficient for my use, I would not even have anough storage if I’d only have 16TB usable, then also, mirror vdev’s doesn’t even protect against 2 disk failures. So just like in the comments in those discussions, mirror vdevs have their place. But not like the article tries to make it out to be
Thanks for the reply, Have you ever ran RAIDZ also? I’d like my 28TB of usable storage with RAIDZ, but if that means higher possibility of pool failure, I’m leaning towards RAIDZ2, what made you choose for RAIDZ2 in the first place?
I have not run RAIDZ yet no. I used to run in a normal hardware based RAID5 with a 6 drive NAS I ran before. Running in RAIDZ vs Z2 vs the pool mirror (or RAID 10) won’t increase or decrease the possibility or likelihood of failed drives. They will simple give you more, or less protection. I decided when bumping from 6 drives to 8 that my media was important enough to me to want to increase to a 2 parity drive setup over 1, giving up another 5TB of storage. I’m glad I did as there has been once or twice when either troubleshooting bad backplains/SATA ports on a mobo, or a failed drive that I’ve ended up having 2 drives out (which would not have been possible in RAIDZ).
I used all WD Red drives for my 6X2TB and 8X5TB. I bought the 2TB’s back in 2012, sold them to my friend when I upgraded in 2014, as of today only one of those 6 drives has started to fail and needed to be replaced. So it really depends on the drives you get too
Thank you all for your replies. I think I’m gonna go for RAIDZ2 for my 8x4TB WD RED’s. I’ve also had my pool fail on me once, when I tried to “expand” my array and didn’t even check if all my new drives worked beforehand… Stupid mistake I’ll never make again.
And regarding the mirror vdevs. I’ve got all 8 bays of my server populated, so when I would want to upgrade in the future. I’m just gonna go down the process of replacing all 8 drives with 8 new drives and resilvering them one by one.
And yes, Backblaze, always interesting to read their year reports. Although with home servers, people will generally go for the cheaper option (shucking often) anyways, just like I did
I originally started a 4x4 array in an R710 (I believe you have an R720 btw as you have ivy bridge xeons vs westmere EP xeons), using a flashed Perc H200 to IT mode (H200 and H310 are exactly the same chip, just different location of the SFF 8087 connectors). I’ve since expanded to 8x4tb in my new build and am running RaidZ2 as I’m in a similar position as you with my Plex library is sitting at about 17TB. While mirrored vdevs are great, it sounds like you’re wanting to just use the same hardware you have and not purchase a bunch of higher capacity drives, so a RaidZ2 may be what you want for now.
