Best way to expand/grow zvol

Operating Systems & Open Source Linux
1

Howdy forum,

The time has come to grow the media server. I currently have 3 drives in raidz1. 2x 16TB + 1x 18TB. I just ordered three more 18TB HDD’s. What would be the best way to grow this? Ideally I would like to remove the 16TB drives from this pool and then use them for something else so that everything is 18TB. It is for media so redundancy isn’t a big concern, will probably just keep it raidz1.

2

As of now it is not possible to grow a raidz1 vdev by adding more drives. This functionality is announced to arrive “soon”, though.

This still leaves a bunch of options - all with pros/cons.

  1. Replace the 16TB drives with 18TB drives. You’ll gain 4TB storage in a 3 drive setup. Use the 16TB drives for something else. Small effect.
  2. Add another zvol. Any type is possible. single, mirror, raidz1/2, … Let’s consider adding another raidz1 vdev with 3 drives. Buy 1x16TB drive, 2x18TB drives. Replace the 18TB drive with the new 16TB drive (to convert your existing vdev into 3x16TB raidz1). Then add another vdev with 3x18TB drives. Total storage capacity 68GB, roughly double read/write speeds. Requires 6 connectors, $$$, increased power consumption.
  3. Migrate to a new zfs pool based on mirrored drives. These are more flexible (you can add remove mirror vdevs from a zpool). E.g. buy a single 18TB drive. Then
  • Create a new pool consisting of the (single) new 18TB drive. Migrate data over from old pool (zfs send/receive)
  • Destroy the old pool. Attach the 18TB drive to the existing drive in the new pool to form a mirror. Add a new mirror vdev consisting of the two 16TB drives.
    => now you have zpool with double the write speed and 33% faster read speeds, 34TB storage.
  1. Build a new pool of 18TB sized drives of desired quantity and migrate data from old pool.
1 Like
3

Add another 3-wide RAIDZ vdev with the 3x18TB. Done. Easy +36TB.

You can do a 16TB mirror and two 18TB mirrors. Perfect capacity utilization and maximum amount of vdevs. I’m a mirror evangelist, so I have to mention that option :wink:

“everything is 18TB” isn’t really important. And +2TB difference on one drive in one vdev is negligible. Just treat it as a 16TB drive, that should alleviate your “purity concerns” :wink:

ZFS will run just fine and doesn’t care. But buying a 18T drive with the prospect of buying more 18TB drives was a good call.

Unless it’s a dedicated backup disk/spare, I don’t see any productive use in not running them in your pool. Pooled storage is what makes ZFS great. All your storage into one pool.

Fragmenting disks and capacity over multiple machines makes all of them slower and you have double the work.

1 Like
4

Unpopular opinion. If you don’t mind risking the loss of the pool a bit (even better if you have actual backups, people should take backups more seriously), then take the 18TB disk offline and use it with the other 3x 18TB disks to make a raid-z1 4 disk pool. The performance will suffer, because of the non-optimal stripe-size / sectors, but if capacity is all you’re looking for, it should be fine.

ZFS-send from the degraded 2x 16TB raid-z1 pool over to the 4x 18TB new raid-z1 pool.

Ranting for a bit. Personally I’d make it a 5-way raid-z1 with 18TB drives, if the budget allows and your hardware can take the additional drive, it’d be the optimal sector split and IMO the best capacity to redundancy risk. You get 80% usable storage, doesn’t get better than this, if you want more redundancy, you start losing a bit on the capacity. And if you want more space, you need to increase redundancy, otherwise the risk is too high, so you lose on the capacity ratio.

2 Likes