So my use case is mainly Heavy IO for my Machine learning training + Upscaling and Video editing work and it seems that my 2x Samsung 850 Pros 512GB in RAID0 is just holding me back now and the fact im running out of storage, ive seen plenty of used Optane DC / Micro Enterprise drives (4Tb) for under 350-400 and wondering if they would offer me a better experience or if i should just buy 4 980 Pro NVME + Raid0 them for roughly the same price?
I have no real clue about enterprise drives and would love some insight
If cheap consumer drives do the job, then this will be the cheapest option per TB. But 4x980 Pro can’t be bought for that 500$ budget.
But when I hear heavy writes, high sustained write speeds and high endurance are the name of the game.
Usually there are two lines for enterprise drives. Read optimized and mixed use. You are looking for the mixed use stuff. And then there is Optane which is 5x as expensive as datacenter SSDs.
I’m getting a Micron 7400 MAX for my server soon (7450 MAX if available, they’re very new and rare). They are really good when it comes to writes/IOPS and endurance. Capacity ranging from 800GB to 6.4TB and they’re also available in M.2 form factor for the smaller capacity variants, but U.3 form factor has higher power limits and performance. Higher capacity == higher write speeds/IOPS
You could add hard drive storage to park your bulky files not currently in use. You might also add a decent NVMe drive as your working area or scratch pad. NVMe is much faster than SATA, it does not make much difference to boot times but much faster file copy.
The best value SSD size is 2TB at the moment and it should not bankrupt you to get one.
Remember the way SSDs work is they have to write large blocks to empty storage. They initially write the data to their own RAM then uncompressed to NAND then in the background inside the device it gets crunched down. When you fill them too full they struggle.
Your system RAM should also be as much as you can afford. After the OS has spread it self out to take advantage of the space it then caches all disk operations which will make a big difference. A computer may appear to have free RAM out of 16GB but it’s actually keeping some free just in case. Put it on 32GB and you will wonder how it fitted in 16GB.
I’ve been using a HDD for cold storage and it’s been fine but it’s the main read and write drives I have that are slowing me down, NVMes are something I’m going to get, I can get 2TB of Nvme now and just expand more in the future due to having the PCIe lanes todo so
RAM won’t really be an issue as I’m upgrading from 64gb Quad Channel to 384Gb Hex channel in about a weeks time so I could utilise a RAMdisk too and have some more bandwidth.
NVMe will help a lot. A RAM drive is the ultimate in speed. Remember that depending on the quality of the SSD they slow down massively after writing say 64GB of data in one copy. Say you have a 100GB file, the first 64GB will write in a few moments then you will be waiting forever for the last 36GB. This is why some SSDs are a lot more expensive. If this sounds like something you do a lot of then focus your efforts on getting an SSD that does not have this problem. I know the Optane don’t suffer this but the cheaper the SSDs are the more this is a problem.
PS, hard drives don’t suffer this, they chug along at their set speed all the time.