I use LVM everywhere. I really like it. I probably don’t use it to its fullest potential, though. However, there is one use-case where I use it in a unique way compared to my other uses of LVM. In this case, I have a LV that spans across multiple drives where I store my games. Once the VG gets close to filling up, I add another drive. Currently, the VG consists of two 1TB PVs; and I have about 244.7 GB left. So it’s time for me to buy a new drive, but it’s honestly a super low priority. Though, now that I am thinking about this, I find myself wondering how LVM manages multiple PVs in a VG. This VG started out with a 1TB drive in 2019. I added the second 1TB drive to the VG in 2021. So, the original drive was almost 100%, and now the ~1.5TB worth of storage exceeds that drive’s capacity. Notably, SSDs last longer with lower levels of utilization. Would LVM wear level the drives as one unit or does it wear-level the drives as the independent volumes that they are? Or maybe put another way, would the drives’ utilization be spread evenly between all drives in a LV, or would it fill up one drive at a time.
Unless you told it to do otherwise, the default would be linear/concatenated:
You could run something like iostat 1 while you access data on the LV and see which drives are being read/written to.
Hmm. Why would someone want to use LVM RAID over mdadm (and vise versa)?
Hot take but I still prefer to use mdadm for RAID for a base XFS or EXT4 filesystem and haven’t found a use for LVM in the enterprise yet.
LVM does make filesystem trim complicated in certain situations. You almost always have to explicitly enable LVM trim support on your linux OS through custom config files, and some SSD’s do their own internal wear leveling and actually don’t benefit from OS trim.
On the other hand, you could get the best of both worlds by using LVM2 for snapshotting, and then lay the f2fs filesystem on top for auto wear leveling on that LVM2 chunk/subvolume
So I will say that the primary reason I chose LVM in this instance was to be able to easily “append” new drives to the VG and LV.
To be honest, I’ve never used LVM for snapshotting, but I know it exists; I just haven’t prioritized looking more into it. My main reason for using LVM is because it makes full-disk encryption easier. The drives in this use-case are unencrypted, however; but I will have to look into F2FS. I also don’t use BTRFS because I’ve been burned by it once… I pretty much use XFS almost everywhere though.
Worth it for the checksumming and lz4 filesystem compression. Not sure if XFS has the same. F2FS also supports native encryption so for your use case you could actually do without LVM I think
(I assume you mean using LVM instead of mdraid, because it’s common to use LVM without RAID on top of mdraid arrays.)
There are certainly pros and cons of using LVM raid instead of mdraid. Well, first of all not instead of, but more like a front-end to mdraid, because LVM internally does also utilize native Linux RAID (i.e. mdraid), but abstracts it away at the slight performance cost.
One of those pros is that LVM RAID in general fares better in case of non-uniform drive arrays. It allows seamlessly extending existing arrays in ways that are much more comples/impossible with raw mdraid. In general I’d say LVM RAIDs are much more flexible than mdraid arrays.
On the other hand mdraid can win in raw performance, i.a. because it’s closer to native hardware (it may directly operate on raw drives instead of going through the extent layer). Here are some results from my own LVM-on top of-MDRAID vs LVM-RAID benchmarks.
As an example of a situation where LVM fares better, consider an array of 4 drives of the same size (say X TB), in RAID10 or 5.
One of the drives fails and is replaced with a new, bigger drive (say 2X TB).
With mdraid you either need to partition the new drive, or you lose X TB of usable space, until all the other drives are replaced. And if you partition, you’re likely to encounter headaches down the line, e.g. when all the other drives eventually get replaced.
On the other hand, with LVM, after replacing the drive you’re immediately presented with the remaining X TB (albeit without redundancy, so unusable for the existing data, but it may be used for something else). After replacing another drive you can start adding new LVs with -m1 and after replacing all of them you can finally just lvextend your original datasets.
Another area where LVM raid beats raw mdraid is flexibility with mixed redundancy datasets. E.g. you create a single array of your NVMes, but you don’t need to have the same level of redundancy on all your data. So your OS gets a raid0 or a raid10, your swap a raid0, your home a raid1 with 2 mirrors and your archive a raid6 because you can sacrifice performance there. All fully extensible and migratable (lvextend, pvmove).