I have been working for years now primarily using my NAS as my work drive (4tbx4tb in FreeNas box). This works well for lighter projects (I do 3D modeling) but recently I have started to get more of really heavy modeling projects from my client.
3Ds Max doesn’t work well in these cases if I try to access the work file from NAS storage so I need to keep the files locally and only back up to NAS or my other external hard drive. This does work but I also tend to get low on space in my internal SSD so I need to start to look at options.
A single 2TB SSD would barely do size wise and would my budget, but with that money I would be able to get two larger HDDS that I could set up to RAID mirror or backup scheme like this. Is this still a valid option?
From what I gathered, the issue is size on your PC and speed on the NAS, right?
Well a single fast HDD will solve all your issues.
Put 8t in your system, free up space from the SSD so you can work comfortably, use if tor local backup, once you are done - move the files from it to the NAS.
If you are afraid of failure, you can use raid or something, but I had more SSDs fail on me than HDDs…
I always use this website to see how different raids will perform with different hard drives. You could compare the read speeds of a Raid1 with a particular ssd you are looking at
I would get the 2TB SSD, it’s just such a huge performance benefit. Then you could use the NAS as secondary storage and swap active projects to the SSD. You could also buy three 1TB drives and put them as a single logical volume.
Pretty much all workstations are moving to SSD / NVMe storage these days, in a couple of years you will be able to replace that 4x4TB with a 4x8TB SSD setup.
Sorry of I missed it but you don’t specify what hardware you are running.
In recent intel could you use optane to accelerate the local hard drives, or storMI for AMD?
Having recently removed all spinning rust from my workstation I doubt i’d go back but you can’t beat the price / performance of a couple of raid1 drives. It’s a tough call for your use case.
@Airstripone Thanks! I run 3950x and 64gb RAM and using Aorus Pro x570.
I bought one 8TB Seagate Ironwolf Nas drive and will maybe buy one more for Raid1 setup as I go.
So you may be able to try that. It may speed up your workload. Make sure you backup first and have your OS on a separate drive.
That board also supports PCIE gen4, so as @wertigon says, wait a year or so and buy a 2TiB NVME drive for cheap then laugh at us puny mortals with our slow sata storage.
Out of interest, what’s your workflow normally? 2TB seems like quite a lot for 3D models, unless you’re storing all assets and textures on that drive?
Is there a possibility of upgrading to 10Gbit networking (either via 10GbE or SFP+) on your NAS and workstation? You’d be working at (for all intents and purposes) drive speeds at that rate, and still keep the benefits of running ZFS on the FreeNAS machine.
@Airstripone True, that is quite interesting. Would StoreMI make sense for work drive setup? It is true that it should be able to keep recent data in SSD.
2TiB NVME gen4 would be sweet, definitely will take a look at that.
@Zavar Hey, thanks for the reply, really appreciate it. Normally I work with 3ds Max, Substance Painter and V-Ray. I do work with very large assets, often 20+ Million polys and very large texture sets. I have dealt with a single Substance Painter project files over 20 Gigabyte. Add two automatic backups that SP creates and I just got 60Gigabyte gone just for one asset, and then we also need to be able to export the data, often at least some maps in 32bit format.
Currently I store these in my main work folder which sits in my main NVME that I boot from, and I use SyncToy to backup that to my FreeNas. The biggest reason for this is that 3ds Max gets unstable when trying to work from network storage, Autodesk recommends officially against this.
Your suggestion of upgrading to 10Gbit networking would really be great for other use cases though, sometimes I render out long animations or such and those files can also start to get heavy really quickly especially if storing the files in 32 bit for later processing.
I use Max for work too, so you definitely don’t have to tell me that haha. For Substance files that big, NVMe is definitely the way to go then. Access times on a pair of traditional hard drives aren’t going to be anywhere near.
Samsung’s 980 Pro series is reportedly launching soon, and they will be considerably faster than any other PCI-e Gen 4 M.2 drives currently on the market, so that could be worth looking at (although I can’t find any info on >1TB drives). I’m not sure how well StoreMI will work with such large SSDs, as its primary use case is a much smaller cache SSD with a large hard drive.
Apparently a replacement for the original StoreMI software is coming soon.
According to the FAQ it should have lauched in July, but i cant see the drivers on the site.
In theory yes, any regularly used objects should be cached and the rest of the data should be transparently accessible off the slow storage tier. How it works in reality for your loads I can’t say, but it should not be worse.
@Zavar Wow cool, 3ds Max comrade! True, with files that big it will lag pretty much just to open a file from HDD. Instead I think I will just keep the disk as backup in local machine, and always work from SSD.
Thanks for the heads up on Samsung 980 Pro, definitely look into it. Should be very fast, right?