I’ve been looking at these non-ZFS alternatives too. If performance needs are not super high or if it’s an all flash rig, LVM integrity RAID is not overly complex to setup.
If it’s not all flash I’ve found LVM integrity RAID (and DM-Integrity) have a big write performance penalty.
- In 7k SATA HDD testing (R1 mirror of 2 drives) I would see a 90% decrease in write performance but only about 15% decrease in read performance.
- In NVMe testing (R1 mirror of 2 drives) I would see a 73% decrease in write performance and about 18% decrease in read performance
But with blazing fast NVMe those performance hits are tolerable and not all that different than you might see on ZFS.
On HDD that turns a 200 MB/s LVM or MDADM mirror write speed into around 20 MB/s which is not so great.
I tried to experiment with layered caching, greatly complicating the setup and found why so many have just stayed with the relative simplicity of ZFS.
You can layer read Caching via Stratis with super high performance on the read side and a comparably simple setup as a similar ZFS pool and file systems. But Stratis doesn’t offer write caching yet in my testing. When they do that’s what I would look at.
In the mean time to get write caching you need to jump through some significant hoops, but it is possible with a custom recompile of LVM2 (bypassing some safety controls).
More info here from some really helpful devs on the LVM GitHub:
I’m worried the custom patch / recompile might need to be redone after subsequent OS updates. But this functionality will eventually come to native LVM as more users test successfully, and no doubt to Stratis as well on the write caching side.
On a side note, if anyone is debating between DM tables caching vs LVM caching, there is a noticable performance boost on the LVM side in my testing. And an even more significant boost in Stratis based caching. I’m not sure why as they’re all leveraging the same underlying tools. But food for thought as you build and test.