Return to

ZFS, ISCSI, Ubuntu, Ideas?

I have an existing ZFS mirrored zvol (4 x 3TB) running on Ubuntu 16-ish which is served to a win-10 desktop via samba. It is used for backups and long-term storage and anything else that I can’t have disappear on me. 10gbe LAN network if it matters.

I was RTFMing and I noticed that PrimoCache (I use it on that win-10 box) can cache an ISCSI target locally to my NVME which now has me intrigued.

What would be the simplest/quickest way to use the ZFS storage space to create an ISCSI target for playing with the PrimoCache?

Assuming that the local caching of the target works well, is there an alternate “fail-resistant” way to store the data and provide an ISCSI target with better performance?

Thanks for any ideas.

1 Like

You’ll have to create a disk image or a zvol and export that via iscsi. I’m not sure if ZFS on linux has built in iscsi target functionality but you can use the iscsitarget package on ubuntu. It’s fairly straight forward and you should be able to find a guide for how to set it up. On windows you use the iscsi initiator to connect to it, which is pretty intuitive to use.

I’ve done this myself actually for game storage


That was kinda what I found, but would raid 5 or something be a better source for the ISCSI? What about cluster/sector/packet sizes or alignment?

Creating a file on a zvol (of a fixed size?) and then making it a target for ISCSI seems horrifically wasteful. Ok for a test, I guess, but sorta kills the whole point of the zfs flexibility. The pre-allocation of a multi-terrabyte empty file makes my skin crawl.

Also, I currently point steam at the network share by default, and then I use freefilesync to clone the game of the month to my SSD which gets cached to my NVME m.2 drive. This prevents any game updates from blind-siding me on the actual game that I’m currently playing, but it’s easy to sync any updates that I do want.

@Dexter_Kane, I just read the whole thread you mentioned, and clearly we were on the same wavelength. Thanks for the thread, and thanks for taking the time to point it out. I missed it in my search prior to posting.

On my 10gbe samba share, ATTO says I get between 600 MB/s and 5.6 GB/s depending on file size, direct I/O, 2GB or 32GB file, 256K or 4MB I/O blocks etc. In real life, games or installers usually get 80-190 MB/s with games on the low end. The bigger the files, the faster it copies when tossing files around. Even when the SSD is the destination, windows writes are criminally slow.

When I first read your 80MB story, my heart sank, but knowing it was much faster on 10gbe, I think I’ll have a go at it. It’s nerly the same as what you did already, so I have a fair chance to get it working.

Thanks again.

You still get all the advantages of ZFS just not at a file system level. I prefer using a large image file rather than a ZVOL as a ZVOL becomes dedicated so you can’t use that space for anything else, an image file will be all zeros so it will compress to an empty file and fill as you use space.

RAID would probably be faster but with a cache on the other end the speed doesn’t really matter. That’s the main advantage I would say of using iscsi rather than samba, although iscsi by itself will be more efficient than samba anyway.