So I conclude that more fast storage is needed and upgrading RAM to infinity and beyond is not a solution anymore.
I need advice on what the best solution might be I run a LGA3647 platform and my bottleneck is currently loading pictures from the Samsung PM9a3 4TB drive. What is a good-to-go setup for more throughput? I am not sure how much of the workload is stride and how much is really sequential. I tried to write everything into an enormous HDF5 file to avoid sequential access to the disc.
So my simple question:
Pci-e 3.0, What is the go-to solution for faster storage within a reasonable amount of money?
Unfortunately you are going out of luck with that first off there are 8TB drives m.2 drives available for between $600-$800. That is the most quick fix I can think of.
Second cheapest is probably going rackmount with a ton of E1.L slots - here you will have 32TB options available today and 256TB options available in a couple of years.
Third, consider u.2 which allows you to do a lot of interesting stuff with NVMe.
At the end of the day it is a matter of how many PCIe lanes you got though, maybe it is time to start thinking about an EPYC upgrade for those sweet PCIe 5.0 slots?
If these are too expensive… Sorry, don’t have any other options that doesn’t involve a ton of jank, headaches and more engineering hours than the other suggestions. Time is expensive too!
intel VROC in your case, I think it requires hardware key to unlock. Should be better than any raid card for nvme drives.
Anandtech review of tech here, you have to unlock this functionality via hardware key and cheaper keys allow only intel ssds. I see them on sale between 100 USD for intel only to 500 USD for no vendor lock. Bastards.
there seem to be no upper limit, just the amount of drives you can actually connect
same functionality on AMD threadripper machines can scale up to millions of iops when you set it up right, detailed writeup here
optimizing your disk access via batching, or massive concurrent amount of threads , to get the most out of you enterprise grade ssd drive
I dont think there are any other solutions here, outside optimizing your workflow.
Personally I would try to maximize memory amount for as little dimms you can set up to get 6-channel or 8 channel config ( no idea which you have) and fill the remaining slots with optane nvdimms.
Yes, with your hw setup, getting an enterprise class, latest gen (PCI gen4) NVMe drive will give you more than 10x the performance of a SATA SSD.
A consumer grade, pro class NVMe SSD (e.g. Samsung 990 Pro) promises to give you the same speed, but will loose steam in extended sessions.
You can get older, PCIe gen3, enterprise class SSDs relatively cheap and even in higher capacity - not quite as fast as the above, but with consistent just below 10x performance of your SATA SSD.
I’d try any of these routes first, and think about raid or other device coupling technologies if these don’t offer enough performance lift for you.
