I have tried to setup a server for our small company using a Ryzen 7 2700X, ASUS ROG Crosshair VI Hero, 64GB DDR4-2666 C16, 256GB SAMSUNG 970 Pro (boot drive), 4x 4TB WD Gold (zpool) and Ubuntu Server 19.10.
KVM QEMU is the hypervisor and I have a standard qcow2 in a dataset of the pool with virtio interface for the VM drive. However performance is kinda bad. I have passed through the Zeppelin USB host controller and I tried to copy a large file in the VM. The write performance at first is kinda good an consistent but at about 18GB it starts to go way down from about 300MB/s to 20MB/s or worse. I already tried changing the buffer mode and using a RAW file instead of qcow2. I also tried adding cache and log devices to the pool but it doesn’t change. Maybe if someone has gone through this already, you could help me out or give me a hint on what I am doing wrong. Here is my pool configuration. Thank you very much in advance
How does your hypervisor allocate the space for the VM disk? Is it set to some sort of thin provisioning where the VM only uses the amount of space that it actually takes up, or is it possible to allocate all the space for the VM disks before hand?
When I ls -l the directory I see the full sizes I have set. But zfs list reveals it’s not actually using that space. I left everything on default at creating the qcow2 file.
Could you try to do some fio/dd tests on the filesystem where you store the VM disks? On the hypervisor itself, not via a VM, just to check if the issue is on the filesystem layer or the VM layer. Maybe it could point us in the right direction?
@cloudstone I am currently doing dd tests with /dev/urandom and /dev/zero … it just takes some time.
@FaunCB On Debian 10.2 and Ubuntu Server 18.04.3 the Windows Server 2019 VM just gives me BSODs and doesn’t run. I don’t know why. So I tried Ubuntu Server 19.10 and I didn’t have a single issue.
dd if=/dev/urandom of=test bs=64k status=progress count=781250
51172147200 bytes (51 GB, 48 GiB) copied, 686 s, 74.6 MB/s
781250+0 records in
781250+0 records out
51200000000 bytes (51 GB, 48 GiB) copied, 686.367 s, 74.6 MB/s
This “low speed” i normal for AMD Ryzen systems. I have no idea why it is so low compared to Intel systems, but I have seen it like this since the launch in 2017.
dd if=/dev/zero of=test0 bs=64k status=progress count=781250
51078299648 bytes (51 GB, 48 GiB) copied, 272 s, 188 MB/s
781250+0 records in
781250+0 records out
51200000000 bytes (51 GB, 48 GiB) copied, 272.576 s, 188 MB/s
you are supposed to set ashift when you create a vdev… i dont think changing it after the fact will do anything,… and why did you select the value 9?
Ok, let me nuke the whole pool then and do it on creation time. I thought it’s 512 but it actually is 4096 so I chose 12 now. Let me destroy and resetup the pool.
