Striped mirror ZFS on proxmox - 10NVME drives, disk space looks strange

having some space weirdness on my server.

the tank is made of 10x2TB Samsung 980 Pro NVME drives in a striped mirror setup.

I have around 600gb on it, and somehow it reads 2.44tb used, i know in Raidz1 this can happen (at least from forums), but did not expect this. can anyone help me get my head around this and how to solve it?

Configured via the UI as RAID10

Thinking that maybe its because of space reservation for the VM that proxmox guarantees and not a ZFS thing, (based on this)

image

One large machine still restoring to it, but the allocation looks ok on the tank view:

image

Disk view:

Based on documentation, looks like the space is promised to the VM. pitty, since in VMWare i could easily do something like “thin provisioning” and only use actual used space.

is there a simple way to achieve that in PVE ?

My ZFS pool only shows actual used spaced and not a full allocation, that goes for both the root pool and storage pool. If you edit the ZFS pool in the Storage menu of Datacenter, is thin provisioning checked?

1 Like

it does now :slight_smile: thanks @2FA !

now, how do I make the change for existing machines to reflect it ? do I have to wipe/redeploy them ? (i have backups) storage currently still shows the full size

also, it says block size is 8k, that’s ok?

With ZFS, changes only apply to new writes so either restore backups or move the disk off the pool and back onto it.

Block size is regarding the volblocksize property of ZFS . It is for ZVOLs which are used by LXCs as their disk. It’s basically akin to sector size of physical disk and modern SSDs have 8k size sectors generally. There can be a lot of nuances with ZFS tuning but since you’re using striped mirrors, it’s pretty straight forward. The defaults are fine for like 95+% of use cases.

2 Likes

@2FA while i have your attention.

my install is on LVM>EXT4 on SSD and the ZFS is its own pool

whats the best way to benchmark the IO ? fio inside the VM ?

Luckily someone has done exactly that how to best benchmark SSDs? | Proxmox Support Forum

Can’t post links. Check out:

pve.proxmox.(com)/wiki/Storage:_LVM_Thin