(UK) Budget system with 512GB RAM?

High-Performance Computing Workstations & Servers
1

I’ve been learning making 3D landscapes as a hobby for a couple years now, and I’m now at the stage where my gaming PC isn’t enough for taking this to a professional level. Even 64GB of RAM can’t really keep up with making an 8192x8192 heightmap (just barely), and now that I’m working with 16384x16384 heightmaps & 65536x65536 splatmaps? Yeah 64GB of RAM is just not cutting it.

Never mind only having 8 cores on a 5800X3D. But there’s a problem.

I’m poor. I had to really skimp and save to get this system after clinging to my i7 6700k with 32GB of RAM. I priced out a quick bang for buck system on PCPartPicker, and an AM5 platform costs 800 quid from 256GB of DDR5 RAM alone!

I also looked at just Epyc platforms with DDR4 but those were astronomical as well with the motherboards being hundreds, and the CPUs are vendor-locked for close to a grand. Which is…Yeah.

Does anyone know where to look for better bang for buck? Ideally I wouldn’t get a CPU with worse performance (though less performance per core but more cores will work fine) since a basic change to a heightmap can take an hour to process through the workflow and tools before it’s in the engine on my 5800X3D, but RAM is a hard limiting factor so maybe I have to compromise.

Any tips?

2 Likes
2

Okay so I would recommend you look into used EPYCs, either in the UK or as import from China.

This is a quick example, but you should make your own research. Feel free to ask if you have questions.

Also you avatar threw me all the way back to my childhood with the feeble files.

P.S. You can go even further back in the Epyc generation but then you sacrifice compute performance since the older EPYCs have many cores but really slow and old ones compared to modern ones.

1 Like
3

8192² heightmap of double-precision floats is 540MB of raw data, so why is 64GB not keeping up for that?

Do you really need it all in RAM at the same time? 3D BVH algos like octtree can quickly assign hot nodes of geometry to threads without needing it all in RAM.

You might be able to get an old Xeon and 512GB of DDR3 for cheaper, but that may well shift the bottleneck to the CPU instead.

Second-hand 400GB P5800X Optane used as swap?

2 Likes
4

I can count on one hand the people that recognize Feeble. Can’t even call it a cult classic as no cult has ever heard of it!

Yeah this is the problem, I found those CPUs but finding a Dell motherboard to pair it with? Impossible for a sane price at least. Then there’s putting these things in a case & a cooler.

Trick is also finding the model numbers. I’ve got a Broadwell system I made from Aliexpress bargain parts for fun so I’m not against buying from China. Especially since China is where all this stuff gets made and assembled in the first place!

The bang for buck on that hypothetical system is insane though with all that RAM.

1 Like
5

Sorry my mistake, easiest solution is to get an unlocked one.

https://www.ebay.co.uk/itm/317045046570

1 Like
6

It’s not just the heightfield that gets processed. There’s all the data generated by Gaea as its processing the graph. My single zone projects have around 350 nodes each and every single one would choke and crash Windows on 32GB. With 64GB and building at 8k, I can just get by without crashes if I close every other application except Gaea.

And at 16k? Yeah-no. And at 64k? Even more yeah-no-so.

I agree about the old xeon thing. I’ve compared the performance in Gaea between my 16 core Broadwell bargain bin server PC vs my 5800X3D, and the 5800X3D was ~350% faster than the Xeon.

7

The Ebay sellers from China with many items sold are usually very trustworthy. I have been ordering 2 Epycs with motherboard from China, one Threadripper too as well as SSDs. Items were worth 4k€ total.

Have a look around maybe you can get a processor with better performance, but these old Epycs are rather cheap and you get 8 memory channels too. DDR4 is still more pricey then it should be but it’s far more affordable then DDR5.

2 Likes
8

Yeah you’re right the cost for 512GB DDR4 is about the same as 256GB DDR5 while Epyc offers 8 channels vs the 2 from AM5. I actually wonder how much of a difference those 8 channels would make, as heightmap software involves constantly modifying gigabytes of data in seconds. Especially when you have parallel node chains so multiple are processed at once, which approaches 10s of gigabytes being modified at any given moment.

This is definitely the way forward. I’d be tempted by Zen 2 Epyc but the latency between CCX units was really bad until Zen 3. Still those prices though. I’ll have to save up for a year to afford this.

How about cases for these server parts? Are they interchangeable with those rack unit whatever things? I’ve only ever built desktop PCs and finding concrete info is actually hard

9

There are dedicated server mainboard, some E-ATX mainboards and some regular ATX mainboards. A regular ATX mainboard should fit in any desktop case. Only thing to look out for is that you can cool the VRMs properly (server cases have very high airflow to cool them). So I’d suggest a board with some VRM heatsink and maybe then direct an extra fan at it.

Other option would be the following:

Threadripper mainboard:

Threadripper CPU:
https://www.ebay.co.uk/itm/276938345841

Maybe you can source something local too but I am not familar with stores or second hand market places in the UK.

1 Like
10

Maybe used workstations? These are old corporate machines with often 8 slots or 16 slots with dual cpu (though probably your software won’t deal well with dual CPUs)… No hassle with racks or server cases. Just an example (haven’t looked too much):

https://www.bargainhardware.co.uk/hp-z840-tower-a-configure-to-order

1 Like
11

This suggestion is more expensive, but Zen3 and you could upgrade to a higher core count CPU if you wanted. Those 5000 series Threadripper go up to 64 cores.

1 Like
12

You could also go TRX40, the Threadripper without the ‘Pro’, but that platform does only have 4 DIMM slots, and that would mean you’d need 128GB DIMMs and I can tell you those cost significantely more then twice as much as 64GB DIMMs.

Sorry, just throwing ideas into the room to give you the info to make the right decision for you when you got the money.

2 Likes
13

The problem with these is the CPUs are much slower at these workloads. I have a Broadwell CPU and it’s around 3.5x slower than my Ryzen 5800X3D despite having 16c/32t. If the workflow didn’t involve an absolute crap-ton of back and forth to make adjustments (good comparison is making music, rendering the output then going back to tweak everything all over again), yeah I’d go that route.

Unfortunately I need to make massive heightmaps with even bigger splatmaps and readjust everything upon seeing the final output until I have an optimal result.

I didn’t know about unlocked Epycs, so I think that’s the route. Just I’ll have to eat lentil curries and rice for the next year to afford it :sweat_smile:

1 Like
14

Actually looking at ServeTheHome, the Z6 G4 and Z8 look interesting. But these Xeon names are insane and a nightmare to find benchmarks for since these names and numbers are not clearly distinguishable.

I’ll probably just buy a CPU off Ebay for cheaper but get a platform from there. What CPU for these workstations approach the 7003 Epycs in performance?

1 Like
15

The 5000 series Threadripper I posted is the same generation as your 5800X3D, why would you expect it to be that much slower?

1 Like
16

I meant the ServeTheHome Intel Xeons, the one linked by @quilt t. Accidentally replied to you, my bad.

The threadrippers could be interesting. But damn threadripper prices. But at the high core count they make a lot of sense on the second hand market. I think with a desktop case it reaches parity with the Epyc. I wonder about longevity, like if threadrippers will retain a high price years from now because of prestige vs the epycs which will get churned out from servers.

Either way this is looking like 2k purchase for superior processing & ram, or 1.5k for a dual intel Cascade Lake xeon build that might have similar CPU processing perf to my current 5800X3D.

Damn why do I have to be poor, it’d be a lot easier and faster to get into the industry if I already had money :sob:

1 Like
17

The ddr4 128GB modules are not much more. 1k vs 0.9k

https://ebay.us/m/2LrtHL

And it gives him expansion capabilities for the future. Maybe he needs an extra 64GB so he can hold his model in ram and still load the operating system and applications.

18

A lot of this stuff is concentrating on the hardware. But I suspect that your limitation is elsewhere. From the description you should be using under 1gb of ram. I don’t know why it would be over that. Can you give some more details on what you are doing?

Also does your current system have an SSD, or are you using a hard disk? When I started messing around with AI, it would take 20 minutes to load a model, then 20 seconds to process it, then 20 minutes to load it again. Swapping to an SSD changed the load time to 6 seconds.

19

These workloads are completely different from AI, and it’s not something common, so I’m happy to clarify!

Gaea is used to create natural landscapes that you create by connecting nodes, with complex formulas and massive node chains. For a 2048x2048 heightmap, you will use around 8-12GB of RAM to create everything from the heightmap, splatmap, and vertex color map. At 4k round 18GB, and at 8k I have to close everything on my system for the workload to not crash from running out of memory.

The pagefile isn’t used by the software, probably because of the reliance on AVX2. I have everything on a fast u.2 intel SSD with 900k IoPs and 3GB/s average transfer speeds, so that’s not the issue.

Then from Gaea, I use World Machine 2 to composite my 2048x2048 zones onto a 16384x16384 canvas. Adding a single zone with a couple mountain stamps uses about 2GB of RAM. After adding around 8 zones and 30 stamps then blending them together on the canvas, I reached the limit of my RAM. So far I’ve used maybe 1/16th of the canvas so far, and I have much more to do!

Then I take the composed 16k heightmap from World Machine 2 then put it back into Gaea for a final erosion pass, then I create the splatmaps and vertex colour maps with the data masks from the erosion and other weathering effects. For a 16k heightmap alone, 128GB should work. But the splatmaps are 64k, which might not even fit in 512GB of RAM.

There is the option of tile modes, but building a 64k splatmap with 8k tiles uses more memory than building a single standalone 8k map. And it takes many times longer & is riddled with bugs that cause the output to be completely blank. So I’d still need more RAM, the process now takes ~3 days to output, and my changes have no guarantee they’ll actually be generated properly.

Whereas if I could fit it all in RAM, a 64k splatmap should take about 12 hours. If the system has multi channel RAM it could end up being a lot faster too.

The problem with DDR4 128GB is the lack of availability. I can’t find those sticks at reasonable prices, with 128GB being around 40% more expensive than 64GB sticks. And remember, I’m poor. My financial situation is I’m permanently disabled from chronic illness, so my only income is from Universal Credit and Adult Disability Payment. The difference in cost will be trivial for most of the community here, but for me that’s massive and takes several months of sacrifice to save up.

20

There’s a really great bundle available from this seller on ebay, item nr 127063599804(just type it in the search field)

the ddr4 sticks are supposedly samsung rdimms. I’m considering getting it myself.

you could talk to the seller to downgrade the cpu’s to eg. 2x 7532’s, they still have 256gb l3 cache, and either upgrade to 16 sticks of ddr4 or stick to 8 and get the other 8 some other time.

1 Like