Getting Started with TrueNAS Scale | Part 2 | Learning ZFS Storage in TrueNAS; Creating a Pool, Dataset and Snapshot Task

Introduction

Welcome to Part Two!
This builds on the first wiki in this series, which you can find here.

After having installed and configured the Basics of TrueNAS Scale, we’ll learn about Storage Pools, VDEVs and Datasets to configure our First Pool and a Custom Dataset. A Snapshot Task will be created as well.

Learning about ZFS/TrueNAS Storage and Terminology

First off, we’ll need to learn a bit about ZFS terminology so it’s easier to follow along.
A lot of ZFS Terminology is used the same way in TrueNAS, but there is some minor differences.
As this series is covering TrueNAS Scale, I’ll mostly focus on that.

I’ll refer to the following graphics throughout this section for visualization:
image

Disks

Storage Devices attached via PCIe, SATA or SAS to your host system are described as Disks.
These must each be physical disks so TrueNAS can perform SMART Tests, keep an eye on the temperatures and spin them up and down properly, they can not be RAID Arrays or the like.

Disks sit at the bottom of the Storage Stack and are what actually stores your data. But except during Pool Creation and Maintenance Tasks, you’ll not directly interact with them on a daily basis

VDEVs

VDEVs, or Virtual DEVices, are the logical devices that make up a Storage Pool.
They are created from one or usually more Disks.

There are different Types of VDEVs for different purposes:

  • DATA: A VDEV used to store the Data stored in the Pool and its Datasets
  • Cache: A VDEV used for L2ARC Cache, optional and only useful if RAM is maxed out
  • LOG: A dedicated VDEV for ZFS’s intent log, can improve performance
  • Hot Spare: A VDEV for spare Disks that can automatically replace broken ones in Data VDEVs
  • Metadata: A dedicated VDEV to store Metadata
  • Dedup: A dedicated VDEV to Store deduplication data (Deduplication is not recommended)

In most simple setups, there will only ever be Data VDEVs in a Pool, which can be configured with the following layouts of Disks:

  • Mirror: Two Disks are Mirrored and Contain exactly the same storage blocks
  • Stripe: Storage Blocks are spread over one or more disks, no redundancy and error correction available with this!
  • Z-RAID1,2,3: Parity Data gets distributed over all drives in the array, with either one, two or three able to fail before data loss.

Often Times, one will opt to create multiple VDEVs of the same Disk Layout if the Number of Disks >10.

ZFS Pools & Root Datasets in TrueNAS Scale

A Pool is a combination of one or more VDEVs, but at least one DATA VDEV.
It stores Encryption and Datasets in an un-encrypted way, so if your Datasets are encrypted the pool stays visible and can be unlocked.

TrueNAS Scale also automatically creates a root Dataset in the Pool, under which a User Creates ZVols or Datasets at their liking.

File Systems / Datasets

File Systems or Datasets are the layer a user or a program will interact with to store data.
Datasets are comparable to Partitions in traditional filesystems (e.g. ext4), but have a lot of added features beyond storing files, for example:

  • ZFS Enrcryption to Encrypt Datasets with a Key or Passphrase
  • Snapshots which reference old Blocks instead of having to clone a whole Partiton to back it up
  • Quotas to Restrict Usage of Storage in relation to the Pool / Root Dataset per Dataset or User

ZVols

Zvols are Block Devices within a ZFS Pool that can be used for Swap Space or VM Disks.

Creating a ZFS Pool

In this Section, we’ll create a simple ZFS Pool out of our Disks with a singular Data VDEV.
Such a Configuration will be used in most simple Homeservers and can be adapted by adding another VDEV to a Storage Pool for expansion or creating a second Pool for e.g. separate SSD Storage.

If you need additional VDEVs you can use this Tutorial as a base and divert as you follow along.

To gest started:

Choosing a Layout for the Data VDEV(s)

At this Point, you’ll have to choose a Layout for your Data VDEV(s).
Reference the Information Above for more Detail on these Options.

As a Rule of thumb, use the following Recommendations:

  • For Maximum Performance: Create One or Multiple ZFS Mirror Data VDEVs

  • 2 Disks: Crate a singular Mirror Data VDEV (1 disk can fail, 50% usable storage)

  • 3 or 5 Disks: Create a singular Z-RAID1 Data VDEV (1 disk can fail, are unusable for storage)

  • 4 or 6 Disks: Create a singular Z-rAID2 Data VDEV (2 disks can fail, are unusable for storage)

When using multiples of the above recommendations, you can add another VDEV of the same type to the Pool for added safety.
For Example, 12 Disks could be configured in two Zraid2 VDEVs with 6 disks each.

Configuring a ZFS Pool with three Disks

Follow the steps below to create a ZFS Pool with a singular RAIDZ-1 VDEV.
We’ll use Encryption and this is recommended in any case unless if your CPU has no AES-NI Support.

Congratulations! You have now configured your first Storage Pool and TrueNAS Scale has automatically configured a Root Dataset:

Creating a Dataset

While TrueNAS Scale has created a Root Dataset on the Storage Pool, this is not of much use yet.
You’ll have to creat a child dataset that will actually be used to share data over the network or be accessed by Software installed on TrueNAS Scale.

In the following Instructions, we’ll create a Dataset to store our home media collection.

Creating a Dataset for Media Storage

We have a Dataset!

Now now Storage Tab should look like this, with a usable Dataset created:

Snapshots

In the last section of this wiki Enty, we’ll create a Daily Snapshot Task for all Datasets in our Storage Pool.

Snapshots reference blocks on the VDEVs that have been modified or changed and therefore copied to a new block, but still exist. Therefore one can roll back easily to an earlier state.

You might wish to create Snapshot Tasks per Dataset if you have different Datasets for different purposes, but in most cases configuring a single task makes sense.

Creating a Snapshot Task for our Storage Pool

That was it! Every night at 12AM TrueNAS will create a Snapshot of the state of your Datasets within the selected Pool / Root Dataset, which can be used for Rollback and Backups.

Conclusion

After following this wiki Entry, you should have a functioning Pool, Dataset and a Snapshot Tasks to go along with it.

In the Next Entry we will talk about installing Jellyfin to actually make some use of our Server.

6 Likes

Part 1: Getting Started with TrueNAS Scale | Part 1 | Hardware, Installation and Initial Configuration

Part 3: Getting Started with TrueNAS Scale | Part 3 | Installing Jellyfin via the GUI + Apps Explainer

1 Like

I only have two 4TB drives at the moment and maybe I can add another one this thursday. I did a mirror vdev now and maybe somehow find a way to convert it into a raidZ1 (or is it Zraid-1?) once I more disks and more enclouse capacity - I only have a pedestrial desktop case right now that I was surprised to realize that it only had two disk 3.5 drive enclosures that I can safely put a drive on.

I also had a spare 120GB SATA2 M.2 SSD. I think i will use it as a vdev cache. Unsure if I actually need more RAM to make it actually worthwhile.

1 Like

There isn’t any way to convert a Mirror into a RAIDZ-1, unfortunately.
If you want to grow the Pool more in the future, adding Mirrors isn’t a bad idea as it’s more cost-effective than adding another RAIDZ VDEV every time.

But also keep in mind that if its data that can’t be lost, Raid is not a Backup, so you could also use the other drive for that.

About your SSD, having a cache can actually lower performance in setups with little RAM.
You could consider to use it for Apps in a Stripe, but only do that if you configure extremely frequent snapshots to the RAIDZ Pool, as a Stripe is generally not amazing with ZFS if you’d like to keep your data.

I’d like to see something specifically covering special devices. I have yet to figure out how they work but I’ve been told that if I add more than one the data is striped across them which means I need at least four in order to have redundant metadata storage, I’d really not like to lose my whole array because I misunderstood how the system was installing the disks.

@felixthecat

First of all I would like to wholeheartedly thank you for these wonderful guides!

Looking forward to a part 3 perhaps? Maybe do an app install?

1 Like

Thank you for the positive Feedback.

I’ve been wanting to do a Guide on installing Docker Containers without TrueCharts via the GUI since TrueCharts already has its own documentation.

1 Like

I was banging my head against the wall for several days now. I’ve reformatted my desktop server again and again because the Nextcloud app was kind of being glitchy and had super slow transfer speeds. Turns out turning on server side encryption really borks the performance hard and causes me grief. I realized:

  • No way I am exposing this server online because of the lack of my technical knowledge will put me and my data at significant risk.
  • This puts server side encryption irrelevant because of the underlying encryption of TrueNAS should be sufficient if the threat model is simple theft/burglary of equiptment.

Also I’ve tried the TrueNAS core variant when I first gave up and it seems to have even basic problems with my Realtek NIC. I briefly considered Debian buster minimal as well as Ubuntu server + Nextcloud snap, but I cant be bothered right now with all the underlying stuff and I dont want to lose the ZFS framework and I cant manually configure that right now.

1 Like

I would recommend you to have a VPN back home instead, it’s much more secure than exposing services on your Server, or at least it’s easier to harden.

Nextcloud might not be using your CPU to accelerate Encryption like ZFS does in this case. You should just leave Nextcloud’s Encryption Off.

That’s because of BSD and you won’t get around it unfortunately unless you’re willing to dig deeper.

1 Like

I’m running into issues setting up the networking part of this setup. When I assign a static IP via the Truenas box settings as per the guide, I can’t bring up the gui in a browser on another machine on my network - it times out. I have a static IP set in the router (sadly my ISP’s router) and if I leave the Truenas box set to IPV4 DHCP, it works. Setting them both to the same IP does not. I have static IPs on my Synology box both on the NAS and on the router so this is odd to me.

Also, setting the router’s IP address as the IPV4 gateway produces an error saying that IP address cannot be found (10.0.0.1 in my case). Looking at the network configuration in the gui, it lists the router address as the IPV4 Default Route.

Another issue is that when I set the http > https flag under General > GUI, upon visiting via https I get the usual warning about unsafe sites and that I am not secure.

I’ve got gremlins! Any suggestions welcome.

1 Like

Hey there and welcome to the forum!

This is because the certificate is self-signed.

As a local server, this shouldnt be a trust issue because you are the one in control and the one who set it up. I believe you can do lets encrypt locally to have a properly signed certificate but that is a longer process and it is, to my limited but likely correct understanding, unnecessary.

As for the rest of you issue, having a router, particularly a pfsense router is very important so that your Synology NAS would not be exposed at the ISP level where it could be hacked.

1 Like

@regulareel is correct about the certificate, you’ll only need a proper one if you’re opening it up to the www, which you should never do with the management portal.

The Rest of your issue might be fixed if you follow my guide now. I just changed two pictures of the networking setup Section in Part 1, where I had mistakenly left out the Subnet Mask for the IP Alias Setting in the Picture.

For me it worked before and after the correction, but this might’ve been your problem.

2 Likes

That fixed it - thank you!!! And thanks also for the advice on the certificate. I will forego that as it does not seem to be necessary at this point in time.

Wonderful assistance - greatly appreciated.

3 Likes