FreeNAS setup (volume options)

Seeking guidance from some of the freenas/zfs/all-round-clever bods!

Looking to build a NAS based on FreeNAS and ZFS. With regards to volume options, I initially was going to choose either RAIDZ2 or RAIDZ3 (depending on the number of drives I purchase, likely 3TB drives). However, I came across this article: ZFS: You should use mirror vdevs, not RAIDZ. – JRS Systems: the blog which has thrown a spanner in the works for me.

Data integrity/safety is most important to me, with performance second, as the data being backed up is vital to me (it cannot be recreated). The NAS would be my third backup (I currently make backups of this data to my main desktop, which is then backed up to an external drive).

Some questions I have:

  1. Which volume option should I pick?
  2. Is there an “optimum” number of drives based on the chosen volume option?
  3. What about “mirrored vdevs” mentioned in the linked article?
  4. Roughly: what’s the procedure when a drive fails? And what sort of time are we taking for the data to be “rebuilt” based on chosen volume options?
  5. What’s the procedure for adding another drive to increase the size? Is this possible?

I’d also like to use the NAS as a plex media server so my desktop doesn’t have to be on for home streaming.

Some info on the HW I’m looking at for the build:

Motherboard: ASRock E3C226D2I LGA1150 (supports ECC)
CPU: Intel core i3-4130T 2.9GHz Dual core (supports ECC)
RAM: Crucial DDR3-1600 ECC memory.

Plus case, PSU etc.

TIA

1 Like

Hi there,
Consider me still a relative newcomer to FreeNAS.

For one thing, you might consider going to LGA1151 for about the same amount of money, since 1150 is somewhat older. 1151 allows more RAM down the road, where you are limited to 64GB with 1150, or what the motherboard specs say. Your choices for hardware seem sound nevertheless. Remember that 8GB RAM is the minimum required, and will probably be more once you go over 6TB of total drives. If you want to do more than storage (VMs and Jails), then that increases RAM required.

Depending on what you choose, you just can't just add a single drive. Not feasible. I'll give my current setup as an example ; 3x 1TB on RaidZ1 = 1,7TB available space.

If I want to upgrade with 3x 2TB, I have to [edited after I read the documentation] put in the three new drives with the 3x 1TB already present, boot the machine and then upgrade. This will replace and retire the 3x 1TB and first silver in the new drives. If I can't for some reason put all three drives together, then I have to retire one drive at a time in the same procedure as a failed drive, which is not the best case at all.

Another upgrade solution is to add another array of drives under your choice of mirror, RaidZ1, RaidZ2, etc. If I understand correctly though, that does not expand the setup seamlessly, but requires to be treated as new network space. Meaning, in my scenario, the extra 6TB is separate from the existing 3TB and can't be mixed as 9TB in contiguous fashion. [Actually, that can be done, but it creates a mixed setup: for the network, it could be a single drive if so desired, but does not change that you now have two RaidZ setups that are unrelated when failure occurs.]

There is a lot of documentation over at FreeNAS if you want to go in more detail, as well as some great How-To's stickied in the FreeNAS forums, including on adding disks if I remember correctly. So don't quote me for a correct procedure here.

FAILURE OF A DRIVE
When a drive fails, you take it out, put a new one in, and it gets silvered in, which, depending on the amount of data and the size of the drive being replaced, can take hours to days, to perhaps weeks. It puts pressure on the remaining drives and hampers performance.

BACKUP
The fact that you are going to have three different saves of all your data is great. Ideally, one save is off site.

If possible, buy different drives from different sources to try and avoid simultaneous or closely timed failures. You could mix WD with Seagate for example. The important part is that the drives are of the same size and similar performance if you put them all in a same pool. In my case, I bought three drives at close to the same time. So maybe I'll preemptively replace a drive a few years down the road even if all is dandy.

There is a lot to learn, so you could do some testing with a first drive and learn a few things, and on a virtual machine too if you prefer, or an old unused rig, which is what I did.

I'll add my thoughts on the mirror setup you linked to.

It's not cost effective to say the least as I learned from experienced opinions, and a waste of ressources. In an enterprise setup, you would have two servers mirrored as one is backup to the other, and in different locations. So in such a robust setup, mirroring is a big waste of money. You end up with a theoretical four times the data plus backup.

Having more than one drive fail is a possibility, especially in a low count HDD setup. Depending on size, you might as well recover your setup from backup, install new drives and restore the data from one of your saves, which may end up speeding things up.

If you have six or more drives, it's very unlikely that multiple drives would fail to the point of rendering the system inoperable if you use RaidZ2 or upwards of.

What follows is unverified from my part, but what I suspect happens.
One of the problems, I think, is if say drive 1A is mirrored on drive 1B. Both fail... Then what. That data is lost, even if you replace them both. Let me explain.

Let's say you start with two mirrored drives of 1TB, then add two 2TB mirrored drives once the two first drives are full on top of the original 2TB. The original data still resides on the first two drives. So if the original two fail at the same time, that data is lost.

Unless FreeNAS redistributes the data across the new drives, which I think is not the case. But the documentation does not mention adding mirrored drives and that scenario, so I think it's unlikely. If it does, it would surely take a long, long time and hamper performance for all that is involved. It's not just copying files over that is involved.

In any case, nothing will ensure data safeguard more than having separate full backups. You have to be ready for total failure of the NAS, which I think you understand. So a full mirror setup is not any more absolutely failure proof than any other RaidZ setup. Failure is still a problem you will have to be ready to contend with, one way or the other.

So once you choose a RaidZ, it's for a long run. And from what I get, the more drives, the better, in that 12 x 1 TB beats 3 x 4 TB. drives, in the purest sense. 6x 2TB is reasonable, and so is 5 x 2TB depending on the RaidZ chosen.