CFD-Multiphysics Benchmark for x86 and ARM (Windows/macOS/Linux)

High-Performance Computing Multiphysics & Simulation
, ,
242

Now we finally have gotten the system built, tuned and ready for running simulations from hereon out!
the new system specs are:
Intel i9 Extreme - 10980XE 18cores 3Ghz stock (2400Mhz cache)
256Gb DDR4 3600Mhz Kingston Fury
Asus Rampage Vi Extreme Omega
Watercooled on Noctuas new NL-LC1-36
Win 11 Pro

10Gb CFD Only: (~66c max after run)
No settings:
67m 15s (there must be something off with HT in such a vast amount of cores)

60Gb CFD+EM:
No settings:
10h 12m 31s

260Gb EM Only:
No Settings:
46m 11s

After extensive amounts of Ram-tuning, Oc’ing, Lapping the IHS of the processor, and changing TIM to Kryosheet we ended up with these results:
10980XE 18cores (HT Off) 4.7Ghz (AVX -2 /AVX512 -3) (3100Mhz Cache)
Cinebench r23 multi - 21318pts
Tighter Subtimings, including tRefi at 32768 Which brought my latency in passmark down from 46ns to 36ns

10Gb CFD Only: (~80c max after run)
No settings:
37m 58s (~44% improvement over stock)

10Gb CFD Only: (~80c max after run)
Dual Run -np8 -mpmode throughput :
62m 43s (~53% improvement)
(~31m 21s / run)

60Gb CFD+EM:
No settings:
6h 38m 18s (~35% improvement)

260Gb EM Only:
No Settings:
33m 43s (~27% improvement)

Final note: I also tried to push the cpu to 4.8Ghz, which gained me an extra 500pts in Cinebench, but only gained me an average of ~5s improvements in the benchmarks at the cost of 30w and 10c extra heat through the system, so i believe cache and further increasing ram-speeds would be where its at

1 Like
243

Those are some pretty big improvements!

I would have expected the CFD-only study to benefit from more clock speed since it doesn’t hit main memory that hard compared to the other two studies.

244

Edit: Might have found the root cause of the degradation: Using company domain logins to ensure IT-security measures :frowning: - the originals runs were done on a fresh install of windows, with a local user at ~37min, the current time is ~1h

Yes! That was also my main reason for choosing this over a threadripper - the price is lower, and it seemed to be an overclocking monster!
(Especially taking the dual-run into account)

Although, for the sake of science and stability, i went on to test on our in-house COMSOL-models afterwards, which seemed to crash the PC - After some deep investigations, it seemed that the Sudden spikes in current-consumption under AVX-loads seemed to bring the 850W power-supply to its knees.

I went on to tune an OC with adaptive voltage-settings, to lower the spikes, and keep the stability in order for us to run 24/7 sims.

This lead to an adaptive-volt overclock with 4.7Ghz and AVX-4/AVX512 -5.
This brings the wattage down to around 350w in (measurable)peaks, and brings my single-core-temps spikes to a sustainable 95c (compared to a steady average of ~65c)

So the limiting factor for the high performance for our in-house model stability seems to be power-supply (i believe a 1000-1200w supply could keep up with the ultra short load-spikes) and the single-core-temp spikes, which i imagine could only be kept in check with a proper Direct-die water-cooled solution, since i already grinded down the IHS a bit (Got 10-15c improvements, and took the hottest core down from ~+25c to +10c in relations to the coldest) even though this IHS is supposedly Soldered to the Die

Despite that, the in-house models are now running twice as fast as on the old i9-9900x - And actually we JUST tried with a dual-run, which has no impact on the sim-times, so it seems that we are capable of running 4x as fast as before.

Although the new OC seems to only be ~17% faster than stock, the new capabilities of the RAM-timings along with more and faster cores, brings our capabilities up quite a lot