Ok, so I started a separate thread on a CPU cooler, only to realize I’m not utilizing the full potential of this place to offer me advice on all parts selection… a mistake I Intend to fix.
I am currently spec’ing out a machine that will be placed in my home office in which I spend anywhere between 24-25 hours a day according to my wife. And that means I REALLY care about keeping the noise low as I do all of my meetings over zoom and development in here.
The way I normally work is with two machines. One I9 12900k + 64gb machine as my desktop, and then a server on which I run most ML and dev workloads.
The current machine that I plan to replace is a Xeon 4699v4 (22 cores) with 256 ECC ddr4 and a bunch of SSDs on an x99 asus was motherboard, with a couple of GPUs in it, which runs ESX. been my dev box for a good 5 years. lives in a phanteks p400a and cooled with noctua fans. My main choke point is I/O and cores as I see it (I’m not a hardware guy) and so the need for a more modern machine is here.
I am trying to stay in the mid-tower cases range since i have room for 2 machines one next to each other and I’d like to keep it nice and discrete, it is my home office after all.
To replace it I started spec’ing a machine like so:
CPU Threadripper pro 5975wx or 5995wx (still debating, but might just pull the trigger on the 64 cores so it serves me for the 5 years growth)
Memory: 512gb ECC 3200 ddr4… you know… because reasons
Storage 1x1tb gen4 NVME for boot,
Storage 4x2gb gen4 for VMs that do data processing
Storage 1x8tb SSD for VMs that don’t matter, or do not require any IO workloads (may use those as an in-esx truenas just to move things around and add more NVMEs later)
Motherboard: ASUS WRX80 sage
EVGA 3080TI that will move to this machine
second GPU to make sure i can passthrough the 3080ti to an ML VM
PSU 1000w platinum something (?)
FANS they will all be noctua, I swear by them.
CASE (?)
CPU Cooler (?)
Challenges are:
Case ? the asus doesn’t like small cases since its super wide)
CPU cooler ? some cases proof challenging on this and I’d like to keep things quiet, but damn WRX80 boards are all oriented horizontally on cpu/ram (god knows why) so I have to be creative and keep noise low. hoping for under 30dbs
Motherboard, will moving to Asrock be a good concession? will I get hurt running ESX on that board ? especially with non-intel NIC’s (i don’t even know if ESX would run on that) .I do not overclock, but I’m an Asus fanboy, always bought those for quality and compatibility
Do I need a bigger PSU ?
an finally, 5975wx or 5995wx… my workloads on 22 cores have been nice, and more is better, especially when I build VMs for various needs like lab stuff, dedicated workload machines, etc… but double the price… my workload is dev/ml and not content creation so Its an interesting dilemma.
have I forgotten anything that I may want to consider ?
I run the asus wrx80 sage in a fractal meshify 2 XL. It does fit, but some of the cable routing holes are blocked.
I use the arctic freezer SP4 4U
No clou/opinion
I guess 1000w is maybe ok, depends on workload. Also make sure it has enough connectors, because the motherboard requires quite a few connectors (ATX 24, 2x CPU 8-Pin, 1x PCIE 6 Pin ← Mandatory, + 2 PCIE 6 Pin (for additional power on PCIE slots)) + plus how many PCIE power connectors your graphic cards need. I’ve put in the corsair AX 1600, thinking just to be sure, but in hindsight I don’t even reach nearly that power usage, so it’s currently overprovisioned.
Does your workload benefit from additional cores, or from higher cpu/turbo frequency?
re #5 - i guess higher cpu frequency for the most of it, but since i plan to run off ESX, then more cores gives me room for more dedicated machines in potential, though i hardly need that.
One other thought on the bubble for me is to stay with a phanteks p400a, replaces the motherboard spacer mounts with taller ones, and not mount the board on the right edges and rather put an acrylic strip under it for strength an/or drill / lower mounts… this is in favor of two things… the cabinet on which it currently stands is 19" deep and all cases that actually support an EATX that wide are longer.
Case? I agree, a case for this motherboard has been one of the hardest things to do. Right now, what I discover. Right now, I am going to get a smaller case and have someone make some custom mounts on the right side of the motherboard. So that I can use a smaller case. To be honest, picking the case takes some time, especially since your desire to make sure it functions correctly, and you have some room to grow.
CPU/GPU cooler? EKWB coolers/Rads and Noctua Fans are working out to be best for me. I tried fans, I tried AIOs, and in the office space - the temps are up there. I invested in an EKWB water cooling system, and by far, the best decision I made. My computer sits right next to me, and I have no issues. Rather I am running full throttle at 100% on the CPU and GPU.
Motherboard, will move to Asrock be a good concession? I am fan of the ASRocks motherboard, but the Asus board is by far on of the best in this extremely small market of about 10 motherboards for this particular CPU. The other board I considered is the Supermicro board but those boards do not come with wifi or Bluetooth feathers.
Do I need a bigger PSU ? For what you plan on doing I would suggest the EVGA 1600 T2 PSU. I prefer to have more power than just barely enough. I have used the system with a few GPU and never had no issues at all.
finally, 5975wx or 5995wx? If you need the 64 core, would you consider the 39975wx?
I am not sure if your going to brand new parts or considering used ones. Right now you can put this system together for pennies on the dollar. With the world of GPU right now, I would jump the 3080 to the 3090’s all day long.
#3 i have no use for wifi/bt… this will run ESX. as a hardcore asus/evga fan (yes) its not an easy decision but dimensions speak for themselves, this will go into most likely a phanteks p400a or p500a. #4 only 1 GPU, (plus a small discrete graphics card to make sure i can passthrough the big one. #5 i invest in a 5 years down the road machine, and seen the delta performance, i will pay the delta. this machine is made to “make money” i.e. i use it for business purposes development and not just for sheer joy, so i can justify the spend. the ROI on it vs cloud spend is within the year for me. that also means that I’m buying new.
The one part i will not be upgrading for now is the 3080TI since in my testing, i am not reaching its full load at this moment and can make the leap in the future.
Now as far as the case. Neither the 400A or 500A case technically will fit this motherboard. I have and do consider each of these cases to downsize my motherboard to. And to do, you will have to plan on creating mounts on the right side of the motherboard. The board you are looking at - is 330 wide. This case can technically only handle 280 mm wide. 99% of the cases suggest A-ETX can work, but when you read the details - the details will get ya every time. So from my point of view, I would make it work. But your board will cover up those factory groumt holes. I have checked with Phanteks on each of those cases personally. So if you are going to make it work, which you can. You will get creative to make it work. Right now, I am working to reduce my case size to something considerably smaller. So at this point - I have researched at least 200 cases.
Case specs.
Motherboard Support
ATX, Micro ATX, Mini ITX,
E-ATX *(up to 280mm wide)
I had to use some PSU power extension cables to route the power connections for the top left corner of the motherboard through the gromets on top. You still have space to the right for either put some water reservoir there or as in my case, mount a lot of HDDs.
Right now, I have the Phanteks Elite Case. I am looking at downgrading to a couple of smaller cases. Right now figuring out the details. I would like to keep my equipment and not feel as if I need to dump the whole system to get a smaller system.
