Hello
I am running a LVM raid 10 on four nvme on a Proxmox 7.2 machine.
The hardware is a Dell R740XD with 12 NVME ports and these disks are using the same controller.
It looks like this:
root@kg-virt01:~# pvdisplay
--- Physical volume ---
PV Name /dev/nvme0n1
VG Name vg-local_nvme1
PV Size <1.75 TiB / not usable <4.34 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 457854
Free PE 0
Allocated PE 457854
PV UUID BnKjnc-9qnR-gOpk-24Iw-u1ew-zrl1-1ZfoD8
--- Physical volume ---
PV Name /dev/nvme1n1
VG Name vg-local_nvme1
PV Size <1.75 TiB / not usable <4.34 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 457854
Free PE 0
Allocated PE 457854
PV UUID RFMNCo-2SRS-5ix2-4Tnn-vBdh-QISv-TiqTEi
--- Physical volume ---
PV Name /dev/nvme2n1
VG Name vg-local_nvme1
PV Size <1.75 TiB / not usable <4.34 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 457854
Free PE 0
Allocated PE 457854
PV UUID d97rc2-TujY-qdBT-1Qgo-Bokz-8Que-JYlKIr
--- Physical volume ---
PV Name /dev/nvme3n1
VG Name vg-local_nvme1
PV Size <1.75 TiB / not usable <4.34 MiB
Allocatable yes (but full)
PE Size 4.00 MiB
Total PE 457854
Free PE 0
Allocated PE 457854
PV UUID i2EDH0-QvRx-1JV8-EEKe-Vrq4-CMzA-gtEhXM
vg:
root@kg-virt01:~# vgdisplay
--- Volume group ---
VG Name vg-local_nvme1
System ID
Format lvm2
Metadata Areas 4
Metadata Sequence No 33
VG Access read/write
VG Status resizable
MAX LV 0
Cur LV 1
Open LV 1
Max PV 0
Cur PV 4
Act PV 4
VG Size <6.99 TiB
PE Size 4.00 MiB
Total PE 1831416
Alloc PE / Size 1831416 / <6.99 TiB
Free PE / Size 0 / 0
VG UUID pRoEAv-CQws-CMB0-VofG-GUYM-E7ka-t605tP
And created like this:
lvcreate --type raid10 --stripes 2 --mirrors 1 -n lv-nvme1 -l 100%FREE vg-local_nvme1
The scheduler on all:
root@kg-virt01:~# cat /sys/block/nvme0n1/queue/scheduler
[none] mq-deadline
So, I am not used to nvme and running it in a local RAID and I wonder how I should measure the performance in a standardised way to see if I get the iops I am hoping for.
What should I do? What could I have missed? Is it wrong to use lvm raid?
All disks are also on the same CPU.
Thanks in advance!