AMD Epyc Milan Workstation Thermal & Mechanical Questions

This topic follows the excellent AMD Epyc Milan Workstation Questions discussion thread, but is intended to focus more on thermal solutions, air flow & mechanical loads.

Official Documents/Lists:
Thermal Design Guide forSocket SP3 Processors
Specs and Manuals
Threadripper Thermal Solutions

Some of the thoughts so far in the forums:

  1. Server boards appear to be designed to be placed in high airflow, low temperature, data center like environments. The fans in them are often the arch-rivals of Noctuas. They may not often care about noise!.
    Just like a pack of wolves, racks of servers howl & whine 24x7, owing to the high airflow fans.
    Also, they generally have a uni-directional airflow. They pull in air from the front of the chassis & vent it through the rear.

  2. Server boards appear to be designed to be placed horizontally like a cake, as they often go into rack mount cases. The two types of heavy weights or pressure that a modern motherboard would typically face would be that of a heat sink and a heat sink. The first one is of the CPU & the second one is that of a Graphics or Compute accelerator card, plugged into the PCIe slot.
    Keeping the PCB laminate(board) horizontally, and then placing those “heat-sink bricks” on top of them, probably distributes most of the pressure to the chassis below the board via the stand-offs.
    But when the Motherboard PCB is mounted vertically, and if a heavy graphics card is plugged in to the PCIe slot, the weight of the GPU’s heatsink, would in-effect try to pry off the PCIe slot from the motherboard. Likewise, a heavy CPU heat-sink, might also create torsive forces on a vertically placed motherboard.


450 grams but (mobo orientation unknown)

Some SP3/TR4 Full Coverage Active Heatsinks with Horizontal Air Flow

  1. Dynatron A39 (Weighs 600 grams!)


  2. Dynatron A35 (Weighs 600 grams!, Appears to have a similar heatsink as above, but a 3800rpm fan)

  3. Dynatron A38 (Weighs 450 grams, inline with AMD’s SP3 spec)

  4. Lenovo CPU Heatsink TR4 280W Part No: 5H40U93000


  5. Silverstone XE02-SP3 (565 grams)
    Productsheet


  6. Supermicro 4U Active CPU Heat Sink Socket OLGA4094 (SNK-P0064AP4)

Some SP3/TR4 Full Coverage (Active) AIO Heatsinks

  1. Alphacool Eisbaer Pro Aurora 360 CPU
    Datasheet, Manual


  2. Alphacool Eisbaer Pro Aurora 240 CPU
    Datasheet, Manual

  3. Coolermaster MasterLiquid ML360 RGB TR4 Edition
    Productsheet, Manual


  4. Coolermaster MasterLiquid ML280 RGB TR4 Edition
    Productsheet, Manual

  5. Coolermaster MasterLiquid ML240 RGB TR4 Edition
    Productsheet, Manual

  6. Dynatron L29

  7. Dynatron L18 (Only dissipates 228 watts instead of the common 280 Watts, has a Top Motor 12000 rpm fan)

  8. Enermax LIQTECH TR4 II series 360mm CPU liquid cooler (Claims to dissipate 500W TDP)
    Datasheet, Manual



    Twice, they were known to precipitate ‘Enermax Slime’ & gunk inside the AIO loop.
    image

  9. Enermax LIQTECH TR4 II series 280mm CPU liquid cooler

  10. Enermax LIQTECH TR4 II series 240mm CPU liquid cooler

  11. Silverstone IceGem 360
    Productsheet, Manual


  12. Silverstone IceGem 280

  13. Silverstone IceGem 240P

If anyone knows about any other AIO or horizonal flow heatsinks which has a cold-plate that covers the whole top area of the CPU, please feel free to comment. I’m not sure if some circular ones are big enough!

1 Like

Some desktop E-ATX cases where the motherboard is mounted horizontally:
This should help with GPUs & CPUs with heavy heat-sinks, rest comfortably without any other special supports.
Looks like not many manufactures make in this stye…
They go around 15 inches wide, which is 4 inches less than the standard 19 inch rack mount :smiley: or a couple of inches less than a standard keyboard!

  1. Thermaltake Core X9 (suggested by @KeithMyers )
    Manual
    GPU - 400mm(with ODD cage)
    GPU - 590mm(without ODD cage)
    Case size : 19.8 x 15 x 25.2 inch
    Has two vertical port holes above the PCI slot




    Modular & stackable…
    image

  2. Thermaltake Core X5 (suggested by @KeithMyers )
    Manual
    GPU - 330mm
    Case size : 18.2 x 14.6 x 21.3 inch
    Has one horizontal port hole above the PCI slot


    Modular & stackable
    image

  3. Thermaltake Level 20 XT (suggested by @KeithMyers )
    Manual
    GPU - 590mm (There is a no ODD cage)
    Case size : 20.3 x 15.5 x 24.5 inch
    Has two vertical port holes above the PCI slot
    (Probably has the same rear as Core X9, but the side, top & front panels are spaced out a bit)


For future use 2

I’m a fan of big horizontally mounted motherboard cases like my Thermaltake Core X9, Core X5 and Level 20XT cases.

No issues with mounting 4 gpus with no concerns with stressing the PCIE slots or mounting big cpu heatsinks. Custom blocks or an AIO eliminates that concern also.

Also plenty of room for tons of fans and radiators. Once you’ve done one custom loop conversion from an AIO, it is real simple to install further conversions.

And you get much better temps on the cpu and gpus and get the max clocks out of the cpus/gpus usually.

2 Likes

Yes, the frames of the Core X9 and the Level 20 XT are the same, only the side panels, front panel and top are different.

The side panels are actually interchangeable between the two models.

The top panels fit the same mostly, just can’t be screwed in since the mounting holes are slightly different,

If you do not care about hight (ie. not a rack chassis), the TR4 version of the Noctua NH-U14 is the way to go. If you need extra capacity, add a second fan on it.

On the weight front, if the heatsink is mounted to a vertical mobo, then it might effectively be an architectural cantilever & with a 2nd fan, it would weigh 1030g + 165g = 1195g or ≈ 1.2KG
That’s 12 trays of canine food!!, pulling on the socket’s screws :smile:

Well, about the dog food, 1.2 kg is the net weight, so contents only. Metal cans excluded.

I think the weight is a complete non issue for TR4/EPYC, the motherboards are high layer count (=thik) and the TR4 socket is a massive steel beast with a robust cooler mounting mechanism. Remember the NH-U14 has a variant for regular desktop CPUs, and a much bigger brother the D15.

Anything short of trying to mail an assembled system, or dropping the case from chest hight should not cause damage to the motherboard. And not all of the weight is packed onto the end of the lever.

There HAVE been failures of the SP3/TR4 socket where the threaded and staked socket nuts have been pulled out of the frame from overtightening the screw or manufacturing flaw.

I’d still stick to AMD’s spec for cooler maximum weight if vertically mounting the motherboard.

This was on a Gigabyte Threadripper board.

1 Like

Happened to come across this vendor provided EPYC Workstation build. This is from the EK Water Blocks company, and it is called
Compute Series X7000 @ USD 27,500/- :facepalm:
Visually, it appears like the board inside is an Asrock Rack ROMED8-2T motherboard.(no info on EK’s site though!)
Looks really neat inside! & and they have not forgotten the GPU support bracket for those Tesla/Quadro GPUs!




Perhaps we can learn something around how system integrators are building workstation solutions around EPYC…

Then, according the some PCI Express Documents on the web,

PCI Express Spec Snapshot
PCI Express x16 Graphics 150W-ATX Specification Revision 1.0

PCI Express 225 W/300 W High Power Card Electromechanical Specification Revision 1.0

Looks like they are trying to say
upto 150W Card = hover under 350 grams or add support if it exceeds.
upto 300W Card = hover under 1.5kg only & “Manufacturers should make
efforts to minimize the card mass” :weight_lifting_man:

Not sure how many GPU vendors specify both the card’s weight & watts consumed, but MSI does:

</tr>
GPU Weight Power
MSI Radeon RX 6900 XT 16G 1520g 300W
MSI Radeon RX 6900 XT GAMING TRIO 16G 1576g 300W
MSI Radeon RX 6900 XT GAMING X TRIO 16G 1576g 300W
MSI Radeon RX 6900 XT GAMING Z TRIO 16G 1581g 300W
MSI GeForce RTX™ 3090 VENTUS 3X 24G 1387g 350W
MSI GeForce RTX™ 3090 VENTUS 3X 24G OC 1387g 350W
MSI GeForce RTX™ 3090 GAMING TRIO 24G 1565g 370W
MSI GeForce RTX™ 3090 GAMING X TRIO 24G 1565g 370W
MSI GeForce RTX™ 3090 SUPRIM 24G 1895g 420W
MSI GeForce RTX™ 3090 SUPRIM X 24G 1895g 420W

What I could observe with MSI cards is that weight hovers around 1.5KG and power hovers around 300W give or take, which is close to the PCI-E 225W/300W Electromechanical Specification 1.0. The SUPRIM ones are probably clocked higher & so would consume more watts & so need a heavy heatsink…

Another EPYC build with Asrock Rome8-2T in the interwebs, where airflow is kept in one direction!

It appears like cooler is a Dyntatron A38 or A26
The GPUs are GeForce RTX™ 2080 Ti TURBO OC 11G (rev. 2.0).
Learned that Gigabyte brands some GPUs that push air only to the rear vents, additionally with the name “TURBO” in them.


Came across the Puget Systems’s TR4 build & found, how they are managing the weight on the socket.

It looks like they are using some custom cut acrylic magic to get some interesting mounting done.


It is interesting to note that the Noctua heat sink has an option on the top to fit in a screw ?
There is something going on with how the acrylic support is mounted with the upper case fan. Not sure if there is a smaller acrylic piece on the mount that serves to convert the vertical hold to a horizontal hold…

Enermax coolers work great, just buy a brand new one every year

1 Like

Will be interesting to see the shadetree engineering that the Optimus Cooling RTX 3090 Kingpin waterblock with active water-cooled backplate users will come up with to handle the 2.5 kg weight of the those cards to keep from sagging or ripping the PCIE slot off the mobo.

1 Like

A good video on how faaaaaast a heat pipe works in transferring heat. You might be amazed! :grinning:

2 Likes

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.