Today I set up an iSCSI target/server on my Debian Linux server/NAS to be used as a Steam drive for my Windows gaming PC. I found that it was much more confusing than it needed to be so I’m writing this up so others with a similar use case may have a better starting point than I did. The biggest hurdle was finding adequately detailed documentation for targetcli-fb, the iSCSI target package I’m using.
I only figured out this out today and I’m not a professional. Please take my advise as such. I did piece a lot of this information from other places but have not referenced all of it.
ZFS zvol Setup for iSCSI Storage
If you’re not using ZFS you can skip this and use the file-based iSCSI backend to store your iSCSI disk as one big file, or research applicable block-based backends for your filesystem. The file-based backend may be very slow, so review your options before using it.
I’m using a zvol as the backend for my iSCSI drive. A ZFS volume (zvol) is a dataset that represents a block device, rather than a dataset/filesystem like we normally use ZFS for. ZFS volumes are identified as devices in the ‘/dev/zvol/’ directory.
I have an existing ZFS zpool with sufficient free space for me to fill my iSCSI drive if I wanted to. I created the zvol like so:
zfs create -s -V 1tb -o volblocksize=4096 -o compression=lz4 h/iscsi-hafxb
Where:
*-V creates a zvol instead of a dataset, and requires a fixed size to be specified. 1tb sets the size to 1024 GB.
*-s creates a “sparse” zvol. This is NOT recommended by the Oracle documentation. I assume that if your zpool fills up, data loss, corruption, or write failures could occur on your iSCSI drive. However, without the -s flag, whatever size you give the zvol will immediately be permanently deducted from your free space on the zpool. This could be a “do as I say but not as I do” situation if there’s important data on your zpool.
*-o volblocksize=4096 specifies 4K sector size. 4096 is the maximum that is supported by targetcli when using the block backend, so it’s a sensible minimum. However, setting this to larger values is advisible to reduce overhead. Feel free to create multiple zvols with different volblocksize and experiment. I’ve found that 256k works well for a steam drive on a pool of HDDs in mirrored vdevs.
*-o compression=lz4 specifies that the data will be compressed using LZ4 compression. There are a few different compression algorithms to pick from but lz4 what I use. The benefits of enabling compression on a Steam game drive will be mixed because many games may not be very compressible, but ZFS seems fairly clever about when it gives up on compressing an incompressible file.
*h is the name of my zpool
*iscsi-hafxb is the name of the zvol being created
If you don’t get an error message, you should now have a zvol at /dev/zvol/{zpoolname}/{zvolname} (e.g. /dev/zvol/h/iscsi-hafxb).
targetcli-fb Configuration
Backstore
This is the annoying part. targetcli-fb is the iSCSI target (i.e. server) package I picked. The man pages and help are very minimal and I figured out what I was supposed to do mostly from trial and error. To make things more complicated, targetcli-fb is a fork of targetcli and the commands aren’t all compatible, so you have to watch what guides you follow. On Debian, targetcli-fb doesn’t seem to ship with a systemd service file or equivalent to get it to start on boot, so we have to make one ourselves.
On Debian Linux, install targetcli-fb with a simple apt install targetcli-fb, then enter the interactive shell with targetcli. Some fundamental commands include:
*ls / works similarly to ls in *nix. The targetcli-fb shell is set up like a small filesystem, which ls / will show.
*cd {path} works similarly to other shells.
*saveconfig saves the current configuration to the default directory.
*exit exits the interactive shell. By default it may also run saveconfig on the way out.
*delete is the rm/delete/remove/destroy command.
*clearconfig true is the factory reset option that will delete everything you’ve set up in the targetcli-fb config. Use when something went wrong and it’s easier to start over.
*get {category} {setting} is how you query the current value of a setting. You can run just get if you don’t know the category to get a list of the applicable categories. You can run just get {category} to list all settings in that category.
*set {category} {setting}={value} is how you change a setting.
It’s important to note that when you’re specifying a subdirectory to run a command in, a trailing slash is needed. For example, in a subdirectory containing tpg1, running tpg1 {command} is wrong while tpg1/ {command} is right.
Create Backstore
zvol Block Backstore
Create a block-based backstore using your zvol as follows:
/backstores/block create name=iscsi-hafxb dev=/dev/zvol/h/iscsi-hafxb
Where:
*name is the name you want to give your backstore. It doesn’t have to match your zvol name, but mine does.
*‘dev’ is full path to the block device in /dev.
fileio Backstore
If you don’t use ZFS then this should be a good catch-all approach. However, the performance may be very slow. This is the command:
backstores/fileio create disk1 /disks/disk1.img 1T
Where:
*disk1 is the name of the backend being created.
*/disks/disk1.img' is the full path to the file you want it to create. I don't know what it does if this file already exists. *1T` specifies that the file will be 1024 GB in size. By default this should create a sparse file, so it doesn’t immediately eat up 1024 GB of space.
Backstore Tweaks
After creating your backstore, there are a couple settings to change, so run the following:
cd /backstores/block/iscsi-hafxb/ if you followed the zvol directions.
cd /backstores/fileio/disk1 if you followed the fileio directions.
set attribute block_size=4096
set attribute emulate_tpu=1
set attribute is_nonrot=1
The block size could arguably be higher for a Steam drive, but this backend doesn’t seem to support any more. This should match the volblocksize of your zvol, and is why I picked 4096 for the zvol.
emulate_tpu=1 enables the UNMAP command which is functionally similar to TRIM for SSDs. This is good to enable even for HDD zpools/filesystems because it also allows you to reclaim the space from deleted files in your iSCSI drive. This is especially important for zvols created with the -s flag (which is not recommended by Oracle, please see above).
is_nonrot specifies whether the pool is composed of HDDs or SSDs. Windows uses this to decide whether or not to make the iSCSI drive defragmentable. With ZFS, even if you are using HDDs, chances are you do NOT want this enabled.
iSCSI Target Setup
Now we set up the actual iSCSI target that will serve access to the backstore to initiators (i.e. clients).
/iscsi create
ls /
Under iscsi you will the name of the iSCSI target that was created. Mine was iqn.2003-01.org.linux-iscsi.dn2.x8664:sn.327f56103dbb. cd to it:
cd /iscsi/iqn.2003-01.org.linux-iscsi.dn2.x8664:sn.327f56103dbb
Now create the LUN that links the target with your backstore:
tpg1/luns/ create /backstores/block/iscsi-hafxb if you followed the zpool directions.
tpg1/luns/ create /backstores/fileio/disk1 if you followed the fileio directions.
CHAP Authentication
If you don't want any authentication you might be done. I would suggest you at least set a username and password. This post covers two approaches:- A simple username and password. This is helpful when you don’t want to whitelist each OS/device.
- A whitelist of initiators, each with their own associated username and password. This is helpful when you want to add an extra (probably weak) factor of security, and don’t want to remember a username.
I prefer #1 for simplicity.
1. Simple Username & Password Authentication
Start in the path of your iSCSI target:
cd /iscsi/iqn.2003-01.org.linux-iscsi.dn2.x8664:sn.327f56103dbb
tpg1/ set attribute generate_node_acls=1
Using =1 makes targetcli-fb use the auth settings associated with tpg1, rather than tpg1/acls.
tpg1/ set attribute authentication=1
tpg1/ set auth userid=username
tpg1/ set auth password=passwordword
userid can be composed of 1 to 12 letters and numbers, and password can be composed of 12 to 16 letters and numbers [REF]. I’m not sure the userid size limit is 12 because with the ACL approach your userid is much longer than 12 characters. Windows’ initiator would freeze and fail to connect when the password was <12 characters though, so stay within 12 to 16.
2. ACL Initiator Whitelist & Password Authentication
By default Windows' initiator assumes the username matches the initiator name, so if you set the username this way in `targetcli-fb` then you only have to remember the password, not the username.tpg1/ set attribute generate_node_acls=0
Using =0 makes targetcli-fb use the auth settings associated with tpg1/acls, rather than tpg1.
Get your initiator name and create the ACL using its name. You can get your iSCSI initiator name if you’re using Windows by running the “iSCSI Initiator” and looking in the Configuration tab. The one on my gaming rig is “iqn.1991-05.com.microsoft:hafxb-w10”
tpg1/acls create iqn.1991-05.com.microsoft:hafxb-w10
cd tpg1/acls/iqn.1991-05.com.microsoft:hafxb-w10
set auth userid=iqn.1991-05.com.microsoft:hafxb-w10
set auth password=passwordword
password can be composed of 12 to 16 letters and numbers [REF].
Last Steps for iSCSI Target
cd /
saveconfig
exit
Now we have to get targetcli-fb to start on boot. I have systemd so I used that. As root:
nano /etc/systemd/system/target.service
Enter the following [REF]:
[Unit]
Description=Restore LIO kernel target configuration
Requires=sys-kernel-config.mount
After=sys-kernel-config.mount network.target local-fs.target
[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/usr/bin/targetctl restore
ExecStop=/usr/bin/targetctl clear
SyslogIdentifier=target
[Install]
WantedBy=multi-user.target
Save and exit (CTRL+X then y)
chmod 644 /etc/systemd/system/target.service
systemctl enable target
systemctl restart target
Now the iSCSI target configuration should be done.
Windows iSCSI Initiator Setup and Unmounting
There’s nothing special that has to be done here, but just for completeness I’ll go over the basic steps:
- In the Windows start menu, search and open the “iSCSI Initiator”.
- In the “Discovery” tab, click “Discover Portal” and add the ip address of your linux server. Port 3260 is the default. Click ok to close “Discover Target Portal”.
- In the “Targets” tab, you should see the iSCSI target name appear. Click on it and hit “Connect”, then “Advanced Settings”.
- In “Advanced Settings” check “Enable CHAP log on” and enter
useridandpasswordas “Name” and “Target Secret”, respectively. If you picked authentication option #2, the default name should match. Click OK to close “Advanced Settings” then click OK to close “Connect to Target” and connect. - If the target shows “Connected”, then in the Windows start menu search and open “Disk Management”.
- You should be asked to initialize a new disk (the iSCSI target). Pick GPT unless you have a good reason not to. If you’re not able to initialize the disk, reopen “iSCSI Initiator” and in the “Volumes and Devices” tab and click “Auto Configure”.
- In “Disk Management”, create a new simple volume (i.e. format) the initialized iSCSI drive. You can keep all the defaults except the “Allocation unit size” should be a multiple of 4096 (i.e. 4K) to ensure consistency with the iSCSI backstore, and whatever volblocksize you selected. My steam drive uses 256k volblocksize, and so I selected a 256k NTFS allocation unit size to match.
. I would consider testing 4k to 256k (inclusive), and sprinkle in however many intermediate values you want to test. You may not find much difference in performance vs the default size.