Homelab Storage Question (ZFS flash upgrades and iSCSI)

I run a 10x4TB truenas array currently, with no flash of any sort, no special metadata etc. I run a dozen VM’s, 2 dozen docker containers, all for fun, none of this is “critical infra” (well, in my mind it is, my house practically stops working when my homelab is down… but I digress. I host truenas under Proxmox (with my HBA passed through ofcourse) and a 10 gig NIC from proxmox to my Ubiquiti switch so I can get 10 gig to my main PC as well.

I plan to upgrade my ATX box to a 45homelab HL15, and with it, overhaul the storage pool. I plan to upgrade to some much alrger Mach.2 SAS drives (likely the 14 TB variant), run 6 of them in a Z2 config, but I am looking to better understand how flash can help me in my use case.

I don’t strickly need more speed, but I like the idea of special metadata vdev’s living on flash, and seeing the price of 1TB SSD’s being pretty reasonable, I figure I can likely grab 3 of these for a Z2 VDEV, and use them for this purpose. What I am less sure about is… should I partition this VDEV into multiple things, them being special meta vdev, L2 ARC (probably not…), and SLOG?

I provide my Truenas VM 64 GB of RAM, and honestly, I hit this with reads so inrequently with my use case I think this is plenty, and I really don’t think adding L2 ARC would buy me much in terms of speeds. I store video and photos on the NAS, but all my active workload is done locally on my PC with SSD’s, I run plex (this is relatively meaingless to the total performance of the array), have some ISO sharing platofrms running, etc.

IMO, the true thing that will make the array “feel fast” would be a flashed based special meta vdev. Thoughts on partitioning the drives to act as L2ARC and SLOG as well?

Onto the question about iSCSI… I would like to provide my windows machine with an iSCSI target for steam games, and Adobe Lightroom catalog and photo storage potentially, but I would want to be able to access the images placed in the iSCSI mount from other devices - what are the options for this? Would something like running rsync on truenas itself allow me to periodically copy the data held in the dataset that is mounted via iSCSI to another SMB mounted location for access by other devices? I am not sure if this use case even makes much sense…

1 Like

The following is based of iSCSI with different enterprise stuff, not TrueNAS Scale:
iSCSI, being a block device protocol, means that it could be possible to be read from multiple clients. The problem is that the one client doesn’t know if the other did a write. Its just a hard drive connected over the network. Thats why I wouldn’t put the Lightroom catalog on it as well. With ZFS snapshots it would be easy to get back a single file on a SMB share. With iSCSI you would have to mount the complete disk.

I would put the Lightroom catalog and photos directly onto a SMB share. For Steam games I would still put them on my computer directly and just mirror the files to a SMB share. I haven’t tried putting games directly into an SMB share, maybe Steam allows this? While I wouldn’t game from it, Steam makes it easy to move games between libraries. So if you want the 2 PB CoD Install ready for when you want to jump back in, you wouldn’t need to download all of it.

1 Like