I’m doing some planning for a new build where I’ll be running Linux as a host OS for several VMs. I was looking into way of building out a software based RAID 1 type array and I’ve seen suggestions to do the mirroring in MDADM, and then use LVM to present disks to the VMs. I’ve seen it mentioned that by doing so you can avoid having to use something like QCOW2 files on an existing linux filesystem and that the storage will be addressed (more) directly by the VM.
Thoughts & Comments?
Why is this desirable? Are you looking for increased performance? Do you prefer managing storage via LVM?
I’m not saying it’s a bad idea, I just want a more complete idea of what you want to accomplish.
I’d start with qcow2, it’s simpler for an admin to move around a couple of files than it is to move around volumes. Theoretically you could get better performance with block devices, but storage devices (even if SSD) are typically slow enough that it doesn’t make much of a difference.
One case where it may make a difference is if the file contents ends up being scattered on the volume underlying the filesystem, but you can probably tune this.
Performance mostly, but I was also thinking of snapshots and how to best isolate one VM from another. Volumes seemed to be more flexible than creating a bunch of partitions or having one single filesystem with everything on it.
Only if you are okay with downtime
To me a large filesystem with qcow files is easier to manage, but it’s whatever you’re more comfortable with really.
To start off yeah its probably easier. The thing is that I’m going to want to run SQL Server a bunch of different ways and I worry about weird interactions between a file system like EXT4 with NTFS existing on top of it. For instance SQL Server likes to generally read/write in 64K extents, so in NTFS I’ll format the file system with a 64K cluster size. I could format the Linux filesystem with 64K cluster size as well, but then everything is running on that and I might want a bit more flexibility to tune those kinds of things based on workload.
Thanks for the feedback!
Don’t quote me, but I think qcow2 allocates 1MB blocks by default, and lvm does 64M blocks. I’m pretty sure both are tunable.
Just remember to set the LVM scan filter on the host if you don’t want to see all the PVS and volume groups inside the virtual machines appearing on the host and inviting you to create a messy catastrophe.
You don’t need to worry much since you will be using a modern host & guest OS. The performance killer for SQL Server reads was disk partition alignment, not NTFS block size. SQL Server uses all sorts of IO sizes and a uniform extent of 8 x 8 KB pages is just one of them, some will be considerably smaller, others larger. The recommendation for a 64KB NTFS block is just a general recommendation but in the real world unlikely to contribute to a huge performance difference. Partition misalignment could however result in a ~30% performance hit on read IO and in older OS’s (Server 2003 & XP) had to be manually set when creating a new partition. Modern versions of Windows automatically create a new partition with an offset that avoids this and I believe all modern Linux partitioning tools will do the same.
I had forgot about those days, fortunately we’ve been off Server 2003 for years, and we run all flash at work.
I was more worried about somehow hitting a corner case when you overlap a virtualized file system over another file system.
Actually sounds like an interesting thing to try and test.
I keep meaning to test hosting SQL Server MDF & LDF files in an SMB 3 share hosted on XFS or ZFS partitions. In theory you can get some benefits (especially with ZFS) but there could be some snags… potentially a good way of ensuring the critical data-files are on a highly resilient file-system, but one that doesn’t cost the earth like some SAN’s.
I’ve done similar testing with data-files hosted directly in Azure storage accounts, that coupled with partially contained databases and Windows Server Core (or SQL Server running on Linux or docker) is a good recipie for interesting things were the VM that hosts SQL can be treated as disposable, any problems you kill it, deploy a replacement, reatach the data-files and back in business. Traditional fialover clusters on Windows server with shared SAN disks can be complex at times - especially if the Windows Admins don’t actually know what they are doing 
Shared storage has always been a pain for us in SQL clusters, which is why if we need HA we use availability groups now. Works really well for us and all of our SQL Servers are VMs.