Run your steam library from a NAS (Break a leg challenge) [UPDATE]


When I built my gaming PC I decided not to use any mechanical storage, and instead only have a single 240gb (this was 4 or 5 years ago) SSD. This was party because I needed to remove all the drive cages to fit a large 200mm fan in the front of the case, but also because I had a NAS server already and figured I could use that for all my storage and only have small SSDs in my other machines.

Obviously 240gb is not a lot of space for games so my method of dealing with that was to install each game to a virtual disk (one per game), this made it easy to move games that I was finished with to the server and vice versa. So I would have maybe 5 games stored on the SSD at a time and I could just move them back and forward to the server as I needed. This worked pretty well but it had some limitations. The biggest limitation was that as far as steam was concerned only one game was ever installed at a time, so half the time when I wanted to play another game I would mount the VHD, load up steam and find myself looking at an 8gb update.

Recently I've been playing around with ZFS and ISCSI and thought I could probably use this to store my whole steam library remotely on the server, so for the last few days that's what I've been working on.

Now as far as I know steam doesn't care if you install your games to a network share or a local disk, it is totally possible to run a steam library off of SMB or some other network file system. An advantage of that is that you can share the library with multiple machines or users. However the performance isn't perfect. I don't have much experience with this method but a few times I would mount the VHDs directly off the shares (rather than copying them across first) which ran okay but I experienced a few issues, such a videos which played during loading screens would often be choppy, as well as some texture pop in issues and other things like that.

So using network shares totally works, but I wanted to try and get as close to native performance as I could which is why I'm using ISCSI rather than SMB.

ISCSI is not like a network share, it's not a file system, instead it allows you to connect a block device (disk) to a computer remotely. Essentially the remote disk appears to be a local disk and it is read and written to exactly like a local hard drive would be.


My NAS server is running Ubuntu 16.04 and I'm using a three disk ZFS RAIDZ array for the storage, the client machine is running Windows 10. While this configuration is going to be specific for what I'm using it can easily be adapted to other systems, freenas and windows server for example will have GUIs for configuring the ISCSI target but it's the same idea either way.

I'm going to create an image file which will be used as the ISCSI target (the disk that the client device will connect to) which will be stored on my RAIDZ array. You could just as easily use a physical disk or any other block device for this. My ZFS array is made up of three 2TB disks, an 8gb partiton on as SSD for the SLOG and an 80GB partition for cache.

	Downloads       ONLINE       0     0     0
	  raidz1-0      ONLINE       0     0     0
	    downloads1  ONLINE       0     0     0
	    downloads2  ONLINE       0     0     0
	    downloads3  ONLINE       0     0     0
	  zil           ONLINE       0     0     0
	  cache         ONLINE       0     0     0

I'll be creating a 3TB image file and leaving 1TB for VM storage and whatever else I'll have on the array. To do that I used dd but there are other (probably better) methods of creating the file.

dd if=/dev/zero of=/mnt/games/daedalus.img bs=1G count=3000

This command will write 3TB of zeros to the file /mnt/games/daedalus.img (/mnt/games is the mountpoint for the ZFS dataset and daedalus.img is the name of the file). This won't take forever, for me it wrote at 2GB/s.

Now that I have the image file I can configure the ISCSI target. install the iscsitarget package if you don't already have it.

sudo apt install iscsitarget

Once installed edit /etc/default/iscsitarget and change the line ISCSITARGET_ENABLE=false to true.

Then edit /etc/iet/ietd.conf everything should be commented out, at the bottom of the file we'll add the configuration for the ISCSI target. I googled my way though this so I've used the standard naming for the target, presumably you could use any name you like but I haven't tried that.

        IncomingUser kane secret_password
        Lun 0 Path=/mnt/games/daedalus.img,Type=fileio
        Alias daedalus

The standard for the target name is the year and month that the target was created, and the FQDN backwards, but I'm pretty sure it would work with any name. The incominguser settings are the CHAP login and password, this doesn't have to be an actual user and you can use anything here, the password however has to be at least 16 characters long. The first target has to be Lun 0, if you were to add a second target it would be Lun 1 etc. The path is to the image file I created earlier and the type is fileio. If it were an actual disk instead of a file you could use blockio instead or just leave out the type setting to have it done automatically. The alias is just a name for the target and can be anything.

Save the file and then edit /etc/iet/initiators.allow

By default this file will contain ALL ALL This means that any initiator (client) will be able to connect. You can leave this as is or you can configure which clients will be allowed to connect to the target.

for me I commented out ALL ALL and added:

So it's just the target from the ietd.conf file followed by the IP of the client machine.

Save the file and start the iscsitarget service

sudo service iscsitarget restart

Now your ISCSI target should be configured and working and you can configure the initiator on the client machine. On Windows 10 go to 'set up ISCSI initiator' in the control panel (or search for it).

In the Target box enter the IP of the server then click refresh, the ISCSI target we configured should apear in the box bellow. Select it and click connect. Click advanced and enable CHAP log in. For name use the user from the ietd.conf file and for target secret use the password. Click okay and okay again and it should connect.

Now if you go to computer management and disk management you will see the ISCSI disk there. Just initialise it, format it and assign it a drive letter like any other disk and it's ready to use.

And that's it. You can install your steam library to the new disk and it will work just like any other hard drive.


On a gigabit network I am only able to get between 50 and 80 MB/s read and write using ISCSI. I believe this to be due to the overhead involved. This isn't so terrible, it means slower load times but that's to be expected going from an SSD to network storage but that is much less than gigabit should be able to do. However I have been unable to improve that. The biggest problem is latency. This is an unavoidable problem with using network storage as just the time it takes to send a packet from the server is several times greater than the seek time for an SSD, but also gigabit ethernet seems to have an IOPS limit of around 100 or so which is pretty low.

What does this mean for games? Well it's not that bad reallty. Load time is obviously a little longer but it's not console long, and most games run fine once they've loaded. I have tested a few and DOOM ran flawlessly, while Bioshock remastered had a long texture pop in time when the level began but was fine afterwards. Mankind divided ran well but there were some times when UI elements would take several seconds to appear, although I had similar issues when running from an SSD.

Here you can see the performance of the ZFS array, network and cache on the server while I'm loading Mankind Divided. The network tops out around 500mbps and the array read won't go above 80MB/s. Even when there are lots of cache reads the network performance remains the same. I'm sure this is due to overhead as when using a 10gb ISCSI connection between the two servers I get the full bandwidth of the disks. It could also be a problem with the Windows ISCSI initiator.

So in short it's a noticeable difference but certainly not a deal breaker.

I believe this problem will be resolved by using a 10gb network. Not only is it 10 times faster and therefore able to deal with the overhead from ISCSI and TCP but it will also be able to better utilise the server RAM and SSD caching. More importantly though is that 10gb ethernet has much higher IOPS than gigabit, I've read something like 1,000,000 or more. That's a huge difference and should be able to deliver that native performance I'm after. Next week I will be able to finish my 10gb network and be able to test that. However I don't want to give the impression that it's totally useless on gigabit, it performs good enough but I will be interested in seeing what difference 10gb makes.

But wait, there's more!

Remember that I said that an ISCSI disk is a block device? Well what can you do with block devices? Caching!

I had a quick look around and found a program called primocache which allows you to use RAM or an SSD to cache other disks on the system. It's pretty flexible and can be turned on or off without having to format or do anything to the original disks. I've been playing around with the trial this morning and it works pretty well, it's only $30 so I may get the full version.

I couldn't shrink the partition on my SSD so instead I created a 100GB virtual disk on the SSD and used that as the cache disk instead. I had to first format it so that primocahce would recognise it, then primocache can format it and use it.

For my configuration I set it up with a 1gb RAM cache and the 100GB SSD cache. The RAM cache is probably useless as the operating system will already use RAM for caching but I turned it on just to play with it. If you have a ton of RAM it may be more useful, and it has the option to reload the RAM cache after a reboot which is pretty cool. I configured it to only do read caching as I don't think write caching is very useful for a steam library, but it has that option.

I've done some quick testing and it does make a difference to the load times of games. Loading up DOOM, playing a bit, then quitting and loading it again resulted in a much faster load time for the second load. In Mankind Divided I travelled between the train stations causing it to keep loading the new areas and the load times improved as I went.

Obviously a cache doesn't know what you're going to do so it can only contain stuff you've already used, but it does give a nice performance boost for the cache hits. I think that primocache plus the 10gb network could result in that native performance that I'm after.


It works, and it's not too bad. Even on gigabit without the cache games are still totally playable. Of course this would work for any kind of application storage and not just games. Now this method doesn't allow for the steam library to be shared between multiple users, although you can share it between multiple machines. While I believe you can connect the storage to multiple machines at the same time this would be a bad idea and will probably result in corruption.

I haven't tried this but if you have deduplication enabled on the ZFS array you could create an image with the combined steam library of multiple users, then make a copy of that image for each user. It won't take up any additional space and each user will just mount their own copy of the image. However the overhead could be too high and not be worthwhile. Alternatively you could create snapshots of the image, this will also not consume any additional space except when new data is written to each image.

Anyway, hopefully that's interesting to someone, feel free to post and questions, criticisms, or suggestions.

UPDATE: Here is the 10gb update that I've posted bellow


Cool project! I was actually playing around with using my FreeNas box as a games library, as I too want to be rid of local mechanical storage. I hope to soon take some time to play around with putting a steam games folder on the nas and seeing how my library fairs running off of that. I'm really hoping I can get origin to play nice with that as well, as being able to free like 200GBs of space being used be all my battlefield copies would be very nice.


Not sure if origin will have an issue with network shares or not, but the good thing about using ISCSI is that the disk is for all intents an purposes a local block device, so origin or anything else won't have any issues installing to it.

I went into the origin application and it did allow me to change the default install folder to a folder on the network drive that I have configured from my FreeNas box, so I can get that far if I do choose to pursue it any farther. Cool to see your experience, if I had my Nas running 24/7 I would definitely look into this more seriously. Alas my 8350 is a hot ass chip and the heat of it in my room is unbearable when I try to go to bed. I need to network more of this house...

My 8350 is the only thing stopping me from freezing to death at the moment :P

I'm going to roast marshmallows on mine at this rate ;P

Cool project :D It's something i've been thinking of for a while but i didn't know how. Nice to see it in practice.

1 Like

I've been playing Metro 2033 for a few hours off and on and I have to say it's working really well. Metro loads fast anyway but PrimoCache is doing a great job of caching the ISCSI disk and the difference is night and day.

The hit rate is nice and high, to the point where if I exit the game, when I come back I don't even see the network light blink meaning that it loads everything from the local cache. I'm pretty impressed by PrimoCache, I've used intel smart response and bcache before and this is by far the easiest disk caching software I've used.

I'm looking forward to trying it out with a 10gb link whenever the NIC arrives, but so far It's working much better than I imagined it would.


This is very similar to the setup that I'm using (minus the caching, nice idea with that btw, never occurred to me to cache files on a local drive). It's definitely usable and is a great way to keep unwanted noise from spinning rust out of your system without selling your left bollock to buy a large SSD. Great job on the write up, very informative.


Awesome! I'm happy to see that you found something to handle caching. I might try to mirror your setup, just for giggles.

@Dexter_Kane, Paving the way in secure network-backed gaming.


I guess I should encrypt the cache too, wouldn't want anyone knowing what games I have! :P

What's the point of the mercury switch if you don't?

Don't forget to tune in next month, when @Dexter_Kane uses thermite to ensure that no one can know.


Fixed it

That was a close one.

I actually found how you did it before steam installing on the NAS fascinating. What the heck possessed you to do it like that originally? Lol

The encrypted NAS? I just thought it would be a fun challenge, to try and make it as accessible and available as possible while still being resistant to various attacks. So I'd come up with ways to attack it and then think of how to protect against it. It was fun and I learnt a lot about bash scripts and stuff which was cool.

This was a much more practical project, kinda sorta.

I got the 10gb network up an running and as I suspected that has sorted out the low throughput. I've seen it get as high as 3gbps and copying a large file from the ISCSI disk will run at 200MB/s plus. Interestingly a lot of the games that I've tested will only read at around 60-80MB/s from the ZFS array and I suspect this is due to random access speed or just the way the games load, other games will get higher speeds and it varies a lot from game to game.

For the purposes of testing I wiped the local cache, so the network speeds reflect the actual amount of data the games are loading. For the first test I loaded up DOOM which wasn't in the ZFS cache, so these speeds are coming entierly from the disks.

It read at around 1gbps, between 90 and 120MB/s. The game loaded quickly, and it was about as fast as I remember it when loading from the local SSD.

The next game I tried was Mankind Divided, I totally messed up the screen shots but the first time I loaded it I was getting read speeds around 60MB/s. This screenshot is from loading it again, so a lot of it is in the ZFS cache.

Pretty fast, it got as high as 3gbps. And the loading time was a fraction of what it normally is. Mankind Divided definitely benefits from fast random access speed, but even the first time the loading time wasn't that much worse than when it was installed locally. The speed of the second load I think came a lot from the RAM cache as before when I was testing it with primocache it didn't load that fast. Also for anyone wondering why Mankind Divided takes so damn long to load, I played it for a couple of minutes (literally loaded the game and walked out of the train station) and in that time the network had transferred 22GB of data.

So obviously the 10gbps network is going to improve load time, but also it has completely solved the weird hangs I was experiencing with texture pop in and stuff not loading when it was supposed to. So that's awesome. So while it is totally playable on gigabit, and the local cache really helps a lot with that, 10gb makes it practically as good as running it local. Any performance loss comes from running the games off of mechanical hard drives rather than the SSD (not to mention the overhead from ZFS and dmcrypt) and not the network itself. So I would say that this project has been a complete success.