ZFS Truenas Scale mergerfs

Operating Systems & Open Source Linux
, , ,
1

I am looking to do some refactoring of my media storage. Currently I have my media stored in a Truenas Scale VM running in proxmox with the HBA hardware passed through to the VM. Current configuration is a 3x12 RaidZ2, so a total of 36 drives, with a mirror SSD for special metadata and a total usable capacity of 230TiB. I have separate datasets created that are all the default 128k record size for all media types. Video Audio Images Text ect. Most of my media is configured in a folder that contains large Video file(s), Images, and Text. What I would like to do is create a new dataset with the 16M record size to store videos in, and a second dataset to store Images and Text in. Keeping the same file structure and using mergerfs inside a docker container on Truenas scale, my goal is to store data types in data sets that are more suited for their type and size. This could also give me the option of creating a SSD dataset and moving the images and text to a better storage medium. But with MergerFS combine both datasets for the file share so all files appear to be in one location, which is required for some software that needs to be able to read the data from a single source. Also from what I see I can also dictate what file sizes go where in the merge as well. I was just curious if anyone has tried to do anything similar and what their experience was?

zfs create -o recordsize=16M tank/Media_Video
zfs create -o recordsize=128K tank/Media_Assets


Media\
  The Martian (2015)\
      movie.mkv          ← stored on the Video dataset (16 MiB blocks)
      cover.jpg             ← stored on the Assets dataset (128 KiB blocks)
      info.txt                 ← stored on the Assets dataset (128 KiB blocks)
2

I don’t really understand how your text gets together with this

but that is probably because I don’t know anything about mergerfs.

You probably already know this stuff but just to be sure I would like to make two comments.

  1. recordsize is a max value. An upper limit. If you have a 16MB media dataset, your movie will use 16MB and the 128kb picture in the same folder will use a 128 record. So you don’t need to separate them.
  2. You might be tempted to use special vdev with a 128kb setting to store the txt and jpg on SSDs. I did that. But there is a catch. Your mkv file could have a smaller than 128kb tail, so part of your movie also ends up wasting space of your special vdev. So I would not bother and leave it a 0 to only store metadata.

This sounds way too complex IMHO. But don’t listen to me, I also think virtualizing TrueNAS is too many layers and against KISS.

1 Like