Latest HPC/CAE/CFD project.
Projected run time on my micro cluster, 4 nodes, 64-cores total system was going to be about 93 days to run out this aerodynamics CFD case on a truck. And that isn’t even remotely close to the meshing resolution that I need, and I’m also writing the data every 20 timesteps (out of 20,000 time steps).
This is about 13.5 million elements, and each result file is around 2 GB.
If I wrote out all of the data (at every timestep), it would consume around 40 TB.
So I stopped the run and cut the number of iterations per timestep down from 100 to 10 and now it should be done in about 9.3 days instead. (Give or take a few days.)
The picture above shows the eddy formation and the wake structures.
(Hardware specs:
Cluster nodes:
4 nodes, each node has: Dual Intel Xeon E5-2690 (v1, 8-core, 2.9 GHz stock, max. all core turbo 3.3 GHz, HTT disabled), 8x 16 GB (128 GB total per node, 512 GB for the entire micro cluster) DDR3-1866 Samsung ECC Reg. 4Rx4 RAM (running at DDR3-1600 speeds due to the number of ranks), Mellanox ConnectX-4 dual port 4x EDR 100 Gbps Infiniband, an Intel 540s Series 1 TB SATA 6 Gbps SSD, a HGST 3 TB 7200 rpm SATA 6 Gbps HDD
Interconnect:
Mellanox MSB-7890 36-port externally managed 100 Gbps Infiniband switch
Headnode:
Intel Core I7-4930K (6-core, 3.4 GHz stock, 3.6 GHz max all core turbo, HTT disabled), 8x Crucial 8 GB DDR3-1600 RAM, Quadro 600, Mellanox ConnectX-4 dual port 4x EDR 100 Gbps Infiniband, four Samsung 860 EVO 1 TB SATA 6 Gbps SSDs in RAID0, four HGST 6 TB 7200 rpm SATA 6 Gbps HDD in RAID0, Broadcom MegaRAID 9341-8i SAS 12 Gbps RAID HBA)
