Slow ZFS resilver performance - would this be a reason to go back to HW RAID?

OR…alternatively – the real answer might be “your mileage may vary.”

(i.e. sometimes you can still get decent performance with HDD pool + RAIDz + block storage, and other times, (on the exact same target), you might not. It really depends.)

Three things:

  1. You are assuming that my root device where I installed Proxmox onto (my 4x HGST 3 TB SATA 3 Gbps HDDs that is a HW RAID6 array via my LSI 9361-8i) was set to be a root zpool.

That is NOT the case.

In fact, the DEFAULT installer, installs Proxmox using LVM partitioning.

To get Proxmox to install using ZFS root, you have to MANUALLY set that, when you are running the installer (which also further assumes that you are booting off the Proxmox ISO, and that you AREN’T installing Proxmox on top of a Debian 11 or Debian 12 installation, which means that Proxmox wouldn’t have been responsible for setting up LVM partition on your boot drive).

You were already EXPLICITLY told:

for the OS drive, just as you were also, likewise, explicitly told:

raidz2 pools are created using the zpool create command.

zvols, on the other hand, are created AFTER the pool has already been created, using the zfs create -V [size] [pool/zvol_name] command (if you want to use the default zvol block size (which for OpenZFS 2.1, would be 8 kiB).

PATENTLY FALSE.

Again, the DEFAULT installer, uses LVM partition for your root/boot drive.

Again, if you want to use ZFS on root, you will have to MANUALLY select that, when you are booting the Proxmox installer via the Proxmox ISO.

(I don’t recall, if you can actually make the Debian 11 or Debian 12 installer, do the same thing where it will install Debian 11 or Debian 12, onto ZFS root. I don’t recall that being an option in the graphical installer for Debian.)

You state that this is the default, but that would be true IF and ONLY IF you had MANUALLY told Proxmox to install onto a ZFS root/boot drive.

If you DON’T set that, the ACTUAL, DEFAULT Proxmox installer, uses LVM partitioning. (which, post install, you can confirm with the commands pvdisplay and lvdisplay).

You were also EXPLICITLY told here:

where the ZFS pool that I am using for the storage of the VM disks and the OS/boot drive are SEPARATE.

Therefore; that already TOLD YOU, EXPLICITLY, that a) I am NOT using ZFS on root) (because you were EXPLICITLY told that I am using “HW RAID6 (hosted by the HW RAID HBA)” and b) the fact that the OS and VM storage have been SEPARATED, tells you, EXPLICITLY, that that’s NOT the default Proxmox behaviour, post initial Proxmox install.

With two lines, you were already EXPLICITLY told FOUR pieces of information:

  1. I am NOT using ZFS on root (the HW RAID6 is the dead giveaway for that).

  2. I am using the DEFAULT Proxmox installation.

  3. And as such, I am using LVM partitioning, which is the DEFAULT, Proxmox installation behaviour, when you are booting and installing off of the Proxmox installer ISO.

  4. I am NOT using zvol (because you were EXPLICITLY told that I am using a raidz2 pool).

With these two statements that I published from the very beginning, these are the four logical statements that you should have deduced/inferred from said two statements.

Clearly, you didn’t because you assumed how the system was set up (which happens most often when people project their installation set up onto that of others, and therefore; build the rest of their statements/arguments around this assumption of theirs where said such assumptions may have little to no basis in reality of the OP who is asking the question and how their system is set up (which can be completely different than how your system is set up).)

IF you MANUALLY set it to install your Proxmox onto ZFS root, then yes, your statements would be true.

IF you don’t (install Proxmox onto ZFS root), then no, NONE of these things:

would apply (IF you DON’T install on ZFS root).

This is true (and I would agree) IF and ONLY IF you installed Proxmox on to ZFS root.

And IF you did that, then congratulations!!! This would be true.

But from my VERY. FIRST. POST., you were EXPLICITLY told that I did NOT install on ZFS root. (Again, the “HW RAID6 (hosted by the HW RAID HBA)” is the dead giveaway for that.)

YOU are assuming that Proxmox was installed with ZFS root.

You were already EXPLICITLY TOLD, right from the get-go, that’s NOT the case.

Therefore; raising this point (about what happens when you install Proxmox on ZFS root) is LITERALLY pointless.

(Again, everything that you need to know about how my system is set up is pretty much EXPLICITLY given to you, in the very first post. Please read it again more carefully the second and third time around.)

Yes, I am well aware.

Thank you.

(Again, you were EXPLICITLY told about the version of Proxmox that I am running, in the first post (which would’ve told you which version of zfs that I am running).)

Or…rather than assuming that I was running zfs 2.2, you COULD’VE just asked.

This is precisely the reason why the English idiom about people who make assumptions, exists.

OS drives: HDS723030ALA640
VM raidz2 pool drives: 4 of the drives are HDN726060ALE614, 1 is HDN726060ALE610, and the rest are HUS726060ALE610

You know that this is COMPLETELY irrelevant to my OP, right?

Again, you already know and have the settings for the OS drive. It’s in the first post.

VM raidz2 pool was created using the command zpool create tank2 /dev/disk/by-id/... with the disk IDs of the 8 drives in question.

Therefore; your key takeaway (in regards to your question about settings are concerned) – if you have any understanding, even a basic level of understanding of how the zpool create command works, then you will also understand the answer to your question about the settings that were used as a result of issuing the zpool create command.

If you have even a basic understanding of what the zpool create command does, what it is, how it works, etc., then you will know what to expect in terms of output, from zpool get all.

If you don’t have this basic level of understanding of what the zpool create command uses in regards to your question of settings, then I would strongly recommend that you read about it in the OpenZFS documentation, so that you can educate yourself, as it pertains to your question about settings.

(because unless I have EXPLICITLY specified a manual overrride via the flag ‘-o property=value’, then that means that i didn’t override any of the settings, and therefore; you can read up on what the default settings are, since you want to know what the settings are.

Find/figure it out (what the default settings are).

(You can literally get this (information) with three lines, in a bash shell script.)

NONE OF THIS, BTW, has ANYTHING to do with my OP.