Yeah you should have done extensive research on the dedupe thing. They way it’s implemented in ZFS isn’t optimal right now. I believe they’re working on some changes but don’t quote me on that. Deduplication is just not something you should be using atm. The only way it’s feasible if you can make sure that the dedupe table ALWAYS stays in RAM or at least is on really fast Optane. 
They only good deduplication methods on Linux, that I’m aware of, are beesd of use on XFS and BTRFS (it’s offline dedupe tho, but fine for me at least) and kVDO on LVM.
there is another way with ZFS…
You can make a snapshot of your Steam Dataset, clone that, use the clone as a second “virtual Drive” OR mount that Clone as an Blockdevice over iscsi 
the benefits are only one “Hardcopy” of your steamlibrary, the clones use near to nothing on space. The only time those clones get bigger are Data that got there from that poin on, like updates or seperate new installs that differ from the main Dataset.
You need to update only one “main Steamdataset” and rinse and repeat.
i wrote an how to over at the Unraid Forum.
1 Like
“Yeah you should have done extensive research on the dedupe thing.”
Can you provide additional resources/citaitons that I should have read, but didn’t?
“They way it’s implemented in ZFS isn’t optimal right now.”
Can you expand on this a little bit more please? (i.e. in what way is it not optimal? And in what way can it be improved?)
I don’t think that dedup is necessarily the problem.
I think that iSCSI is actually, quite possibly, the bigger issue here.
The reason why I say this is because I actually created a TrueNAS Core VM, and tried to make that handle the iSCSI portion of this, and it worked for a little bit, and then the whole thing just died, when I was trying to install games onto said TrueNAS Core VM (which also was responsible for being the iSCSI host/target).
How do you check if the dedup table is and stays in RAM? My system does not have a NVMe M.2 slot for an optane module and to the best of my knowledge, Xeon E5 v4s do not support PMEM100 nor PMEM200, but please feel free to educate me if my information is out-of-date.
Would you be able to point me in those directions?
My preference would be “live/online” dedup so that as the games are being installed, it is being de-duped “on-the-fly”. But if that’s not possible, then I’ll have to play around with offline dedup to see whether that would be a feasible alternative proposal/solution, for this application.
My understanding with this approach is that it assumes that the games that are installed on system A is IDENTICAL to system B no?
Right now, the way that I have it set up is that I create a 10 TiB ZFS dataset, with properties compression=on and dedup=on. From there, an iSCSI target utilises this dataset and the Windows clients mount this iSCSI target, format a 2 TiB partition of the 10 TiB “block”, quick format it as NTFS, and proceed with the installation of Steam games, where the initial setup of the games run concurrently between all of the clients.
So that’s how I am applying these technologies.
My thoughts with this procedure is that I can’t get clients B and C up and running until client A is complete, so that it can be cloned, and then mounted over iSCSI.
Please correct me if I am misunderstanding the deployment procedure as described.
Thank you.
Nope
system A has 100 Games on it
do a snapshot.
Clone it.
Blockvolume it.
iscsi that Block.
System B has now 100 Games installed for basicly a few MBs.
If system B’s Steam account has less or other games available this account can use all bought games installed AND it can install the one system A does not have BUT then
the ISCSI Block Clone will grow of the size the other games have.
Say Accpunt A on System A has 100 Games 5TB Big BUT does not have Doom 4
Say Account B on System B has 100 Games 5mb big BUT installs Doom4 then it has 5mb + size of Doom 4
and so on with new updates, say 5mb + Doom4 + new updates and so on
the best aproach ist Steams Family share so Sytem A can install ALL Games available then clone that.
Or atleast download all Games from Steam Acc A than switch to B an install the rest from there
here see games 4.56 TB
games.clone 4.50TB
but effectively used on the Array 4.56TB
2 Likes
Thank you.
For this to work though, given that you have different games on system A and B, where the clone will only have the delta between A and B.
But if you want to start B off with the same number of games as A, then A will need to be set up first, then cloned, and then you can start setting up B, correct?
(i.e. the configuration runs somewhat sequentially (except for the deltas))
In your example, is system A also an iSCSI block device? Or do you use a different protocol?
I wonder how I would clone the snapshot in Proxmox and/or using the ZFS command line tools.
(Given that I am not using Unraid, so I am going to have to “translate” what you are doing with Unraid to Proxmox and/or the ZFS CLI.)
This is an interesting idea.
Thank you.
(sidebar: Can you also use this for non-Steam games as well? I would think so, but I figured that it wouldn’t hurt for me to ask the question, just to make sure.)
zfs clone tank/filesystem@snapshot tank/cloneIt’s basically like snapshotting, but more like a fork that shares the same blocks.
You can also “promote” the clone to be a fully fledged filesystem via zfs promote, so you can delete the old/forked filesystem if you just want to keep the former clone or swap their relation.
I do think that clones are great for these things, albeit a bit unconventional, and thanks @Domrockt for pointing that out . But better than SMB/NFS shares or using multiple zvols and dedup. TBs of deduped datasets is very expensive.
iSCSI and multiple initiators is not that easy. And NTFS doesn’t help either.
it should be mostly the same commands.
it can be all you want. in my case i have an Win 11 VM for Moonlight streaming to my Living room Apple TV.
So what i did :
Unraid Server:
I created an ZFS Dataset 13TB and mounted that to my VM as a secondary Drive.
Installed Steam, Epic and so on… Installed ALL games with every launcher. Then logged to my other Launcher Accounts i own and installed all the Games too.
that took a while.
then i shut down my VM and made a snapshot, cloned that snapshot, made a blockdevice out of it and mounted that over iscsi to my Main Rig.
Main Rig:
See’s a 13TB Drive with all the games on it, then i need to add the librarys to the launchers and iam good to go.
No
sure all Games all Data its a Clone. Some launchers needextra steps to regognice the games but all works fine.
the how to
what i did
1)make an zfs array, how you like it. myne is named zfs
2)go to the Terminal and create an Dataset with this command
3)zfs create -s -V SizeYouWantG -o volblocksize=blocksizeYouWant zfs/nameOfYourDataset
example: zfs create -s -V 8192G -o volblocksize=4096 zfs/games
4)than you create an ZFS Blockdevice, still in the Terminal enter
5)targetcli
- /backstores/block create name=NameOfYourBlockdevice dev=/dev/zvol/NameOfZFSPool/NameOfYourBlockdevice
example: /backstores/block create name=games dev=/dev/zvol/zfs/games
- cd /backstores/block/NameOfYourBlockdevice/
8 )set attribute block_size=4096
9) set attribute emulate_tpu=1
10) set attribute is_nonrot=1
-
cd /
-
exit
-
Now you can close the Terminal and go to the ISCSI Plugin and create an ISCSI connection to your Rig.
-
Install all your Games on that drive
-
whan all games are installed we make an snapshot in the Terminal with the following commands
-
zfs snapshot NameOfYourZFSPool/NameOfYourBlockdevice@HowYouWantToNameYourSnapshot
example: zfs snapshot zfs/games@allgames
than we want to clone that
zfs clone -p zfs/games@allgames zfs/games.myclone
then again into
targetcli
/backstores/block create name=games.myclone dev=/dev/zvol/zfs/games.myclone
cd /backstores/block/games.myclone/
set attribute block_size=4096
set attribute emulate_tpu=1
set attribute is_nonrot=1
cd /
exit
and now you can mount that “cloned Blockdevice via ISCSI” dont forget to create an individual Target per VM/Baremetal/Rig
when you have new games installed than just go to the Terminal
targetcli
cd /backstores/block/
delete NameOfYourBlockdevice.myclone
cd /
exit
zfs destroy YOURPOOLNAME/NameOfYourBlockdevice.myclone
zfs destroy YOURPOOLNAME/NameOfYourBlockdevice@HowYouWantToNameYourSnapshot
than start over @ 16) rinse and repeat
next scenario, you bought another 2TB of SSD storage just add it to your Array as usual than enter
zfs set volsize=newSizeofPoolG NameOfYourPool/NameOfYourDataset
example: zfs set volsize=13000G zfs/games
now you can add your new storage to your iscsi storage in Windows
Thank you.
I think that some of the commands might be Unraid specific (e.g. /backstores/block), but I can at least use the idea of it.
I think that one of the most significant, operational differences between this and using ZFS dedup is that with ZFS dedup, I can install the games on multiple clients simultaneously (so that they can be up and playing games as each install instance (that is getting deduped by ZFS)) completes whereas with this method, you would have to wait until ALL of the games are installed before they can be made available to each additional client.
So, if you have a large library of games, then you spend the time to install it only once, but other players/clients might have to like wait a day before they will be able to play with you (whereas, you can start playing as each game is being installed).
re: Your earlier comment about ZFS dedup not working properly
That might be more of an Unraid implementation of ZFS moreso than ZFS itself.
It is my understanding that Proxmox has added ZFS sooner than Unraid, and I think that TrueNAS Core has added ZFS support even earlier than that.
That’s an Unraid specific thing.
I might have to test this procedure out.
Thank you.
Thanks for the setup instructions, I just followed them and it worked straight out of the box once I’d figured out that my ZFS devices were in /dev/zvol
I do however seem to have a performance problem though - I’m connecting to my Proxmox iSCSI target from Windows 11, and I’m saturating 10GBe on reads (1.2GBs-1) but only getting ~50MBs-1 on writes to the all-SSD iSCSI target. Are there any tunables I need to play with?