Setting up a ZFS pool (for the second time)

Operating Systems & Open Source Linux
1

I have a ZFS pool that I created two or three years ago. I’ve always sort of felt that it just wasn’t “proper,” that I had never really set it up right. I’ve got at least one failing drive, possibly two, so I figured now would be a good time to start over.

The pool consists of eight 2TB drives. The drives are individually LUKS encrypted, and the ZFS filesystem is inside that. When I set the pool up native encryption wasn’t a thing with ZFS on Linux. So, that would be the first thing I’d like to change.

I also didn’t do anything with VDEVs. I also don’t understand the concept, so if I want to go that route I’d have to figure that out. I’m not sure if it’s something I should do or not.

I’m pretty sure it’s just RAID-Z, which I’m fine with. I’m using about a third of available space. I may do a fresh install of Fedora 28, or just do an in place upgrade. Not sure yet.

I can provide more information when I get home tonight. I need to do a final backup and verify everything is there, then I’m good to go. I just wanted to start the topic now and get the attention of the fine people who are knowledgeable with ZFS, because I know there a few around here.

2 Likes
2

So here’s the status:

[admin@homeserver ~]$ sudo zpool status
[sudo] password for admin: 
  pool: homeserver_share
 state: DEGRADED
status: One or more devices has experienced an unrecoverable error.  An
	attempt was made to correct the error.  Applications are unaffected.
action: Determine if the device needs to be replaced, and clear the errors
	using 'zpool clear' or replace the device with 'zpool replace'.
   see: http://zfsonlinux.org/msg/ZFS-8000-9P
  scan: resilvered 0B in 0h0m with 0 errors on Thu Jun 21 17:40:03 2018
config:

	NAME                  STATE     READ WRITE CKSUM
	homeserver_share      DEGRADED     0     0     0
	  2tb_d01_crypt       DEGRADED     0     0     0  too many errors
	  2tb_d02_crypt       ONLINE       0     0     0
	  2tb_d03_crypt       ONLINE       0     0     0
	  2tb_d04_crypt       ONLINE       0     0     0
	  2tb_d05_crypt       ONLINE       0     0     0
	  2tb_d06_crypt       ONLINE       0     0     0
	  2tb_d07_repl_crypt  ONLINE       0     0     0
	  2tb_d08_crypt       ONLINE       0     0     0

errors: No known data errors

And here’s list:

[admin@homeserver ~]$ sudo zpool list
[sudo] password for admin: 
NAME               SIZE  ALLOC   FREE  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
homeserver_share  14.5T  4.31T  10.2T         -     8%    29%  1.00x  DEGRADED  -

Any other pieces of info that would be helpful? Currently backing up data and will start wiping and preparing drives in an hour or two.

Edit: Oh, and I have a spare 120GB SSD handy. Would that be useful for a cache? Currently all this fileserver does is store files and plex. I may get into blender / video stuff, and would use the storage for that too, but I don’t have any kind of timeline.

3

Whether to (or even if you should) actually use the ssd for caching will probably depend on how you use the pool.

Are you doing any performance tests on the pool after you rebuild it?

From what I’ve read on different site, most people dont recommend using l2arc until you have maxed out the RAM on the motherboard.

4

Well, the system has 16GB RAM (and that’s all it’ll ever have: 2 DIMM slots and I ain’t buying 16GB modules ever). Every time I ssh in the memory usage is pegged. In an earlier iteration of this fileserver I actually limited the ZFS RAM usage so it didn’t take up all 64GB that was in that system.

I don’t know about performance. The way it is now can nearly saturate gigabit ethernet, so there’s that. Eventually I’d like to go to 10G, but again no timeline.

Does anyone know if the version of ZFS that’s included with Fedora (28) (zfs-dkms?) has encryption included? Or do I need to do something else?