Is there a way to exclude certain datasets from having their metadata placed onto the special device?
I have some data that is comprised of folders with anywhere from 20-400 thousand files in them, but none of this data needs to be rapidly accessible. A lot of it won’t ever even be accessed before it is deleted, and when it is accessed, it’s primarily via a script that runs quietly in the background with a bunch of triggers to suspend itself if much of anything else is running on in the system.
Some spitball estimates with sample sets of data indicate that the metadata here could easily balloon to half a TB in various scenarios, possibly higher, and I’d much rather have that as free diskspace for small blocks for some performance data sets in a hybrid pool. Yeah I could buy bigger SSDs, but the cost starts getting rather silly when you factor in the fact that I’m essentially manufacturing the problem.
The alternative appears to be the more classical approach of running with two pools (one nvme and one rust), possibly using a separate L2ARC device. The issue there is that a L2ARC device requires warming, which means it won’t really do anything for some workloads. Not to mention that L2ARC wastes a bit of memory.