Return to Level1Techs.com

TrueNAS Core/ FreeNAS Do Not Expose Dedupe Savings in GUI

With TrueNAS Core being released soon, and the addition of SPECIAL VDEVs we now have the ability to use a dedicated piece of hardware to store our dedupe tables. The Dedupe table in ZFS is stored in RAM, and the more files you attempt to dedupe, the more RAM you are using just to store the metadata necessary to know where the individual blocks are that have been deduped.

In prior versions of the operating system, what would happen when you ran out of room in RAM to store the dedupe table is the ZFS would then put that data on your main pool in the same VDEVs your files/vms/etc are stored…wasting I/O. Now they give you the ability to put that on a dedicated VDEV (must be a mirror, preferably NVME SSDs or Optane).

The problem is that they don’t expose the savings in space in the GUI at all. They factor in the space savings from compression, but not from dedupe. You have to dig into the CLI and run the zpool list command to actually see how much space you are using in reality. This makes it challenging to leverage the space savings gained in your pool, because the GUI doesn’t actually tell you how much space you are using!

If this is something you would be interested in seeing, please upvote my suggestion on the IXSystems Jira :slight_smile:

https://jira.ixsystems.com/browse/NAS-106606

Isn’t the dedupe table backed by main pool storage too, because Ram is volatile?

They can’t rebuild it from ram, if there is an outage/system down/reboot?

And would the new Specail vdevs just replace the pool side of the dedupe table, or combine them to reduce memory useage, as well as main pool I/O?

1 Like

Yes, exactly it was always traditionally written to the main pool storage and read back from there after a reboot, that hasn’t changed really, it’s just reading it back from the dedupe drives. You still are going to read from RAM first, and it spills over into the dedupe drives now instead of the main pool so it makes the penalty for having not enough ram less painful.

All I am saying and complaining about is that TrueNAS doesn’t tell you saved any space so it looks like it isn’t deduping your data even though it is…it was very confusing.