I thought I would post this system I built from spare hardware as we moved to H100/H200 powered systems and got to take some hardware home. The A800s were air cooled but I was frustrated with the noise and lack of control the WRX80E has for the fan control from the OS. I was basically using a Delta 80mm fan to cool all three cards. Alas here are the details…
OS: Ubuntu 22.04.5 LTS x86_64
CPU: AMD Ryzen Threadripper PRO 3955WX s (32) @ 3.900GHz
MOBO: WRX80E Sage Wifi
GPU: NVIDIA A800 80GB
GPU: NVIDIA A800 80GB
GPU: NVIDIA A800 80GB
Memory: 257545MiB
Radiator: Heatkiller htsf2 4x120mm
Distroplate: Alphacool ES Distro Plate Parallel C5
Pump: Corsair Hydro XD5 Pump
Tubing: EK 10/16 Soft Tubing
Fittings: Barrow
Water blocks: Bykski CPU-RAY-MK-M / Bykski N-TESLA-A100-80G-X-V2
Case: Fractal Define 7 XL
Lessons learned:
Few things I had to deal with are that the WRX80E flags a HW slowdown pin on Nvidia GPUs so I had to cover that pin on all three GPUs. Additionally I planned to put this radiator up front but had to pivot to placing it up top. Cables for the PSU were not long enough and had to be routed down instead of through the back. All grommets for the side were covered as well.
Things that could additionally be better:
I would like to have a 20A breaker so I could have used all 4 of the GPUs I had. Additionally, fitting more radiators would have been possible (360 up front) and I think that would have made a sizeable difference here. Under load I am seeing temps in the low 60s with everything loaded running a fluid sim. Maybe I should have shelled out for NV link bridges but I just don’t want to throw money at them yet as I am sure the prices will crater for them in a year.
