"Datacenter in a box" - any caveats around Epyc 7742 build?

Hi everyone,

I am theory crafting proxmox server build around 64c epyc cpu from 7002 series - 7742 specifically.

My usecase is personal and professional learning/proofing platform on a budget. I am getting into a lot of automation lately and having virtualization host where I can run simultaneously:

  • redundant kubernetes cluster
  • redundant ceph cluster
  • redundant reverse proxy
  • redundant database servers
  • windows AD or equivalent
  • windows DNS
  • windows ADCA for local cert issuance

and additional services as needed without having to leave single machine would be very useful. 64 core cpu might overkill, but they got very cheap lately. Memory is decently priced too, so I can get 4GB/core (8x32) or even 8GB/core (8x64) without braking a bank.

Normally this would be completely out of my budget, but I already own used supermicro H111SSL-i with 16 core epyc 7302 and 8x16GB 2133 ecc ram as a kit.

Replacing cpu and memory to upscale it is affordable enough. For extra ~~1000 USD I could build 64core machine with 512 GB ram and thats very tempting. Or small dick syndrome, IT edition.

Does anyone have experience with this setup? Any surprising or devious pain points?

Points of interest:

  • cheap memory is slower than officially supported optimum, 2133 vs 3200. Is it worth paying extra and what is power consumption tradeoff here? I doubt I would benefit from increased throughput, but better know it before ordering anything.
  • 64 core cpu might stress board power electronics so that extra cooling might be necessary, is instability common?
  • 7003 series 64 core cpu droped in price as well, but they are more expensive and require new pcie 4 capable motherboards. Those are extremely expensive, even more so when I bought the original kit. (supermicro H12 series)
    • I do not need pcie 4, and given the price is not worth extra investment even for nice to have futureproofing

Supplemental data:

  • idle power draw is irrelevant, since I intend to power on system remotely only during active work. It will be powered off otherwise. Its purely intended for workload that I build, test, break and then cleanup. Any critical services would be run on separate power efficient proxmox host (ucff ryzen hp elitedesk ftw)
  • original purchase was meant to replace old xeon e3 1225 v3 nas system, but idle power draw of 7002 series was bad enough to nix it. Nas runs 24/7, so it would get expensive really quickly
  • all part will be second hand and imported from china via ebay, I have good experience with specific seller. Financially absolute no brainer, there is no sane alternative to it.
    • I would prefer getting used ThinkStation P620 with Threadripper PRO 3995WX , but those things have no secondary or sanely priced market.

Prices I am basing this on:

Part Ebay link Purchase price Current price notes
7302+H111SSL-i + 8x16GB 2133 ECC Pardon Our Interruption... $577.80 $1,025.24 already owned
Epyc 7742 unlocked Pardon Our Interruption... $499.00 cheapest non oem article
8x64 2133 kit Pardon Our Interruption... $656.00 slow but massive, cost efective
8x32 2133 kit Pardon Our Interruption... $464.00
8*32 2666 kit Pardon Our Interruption... $659.60 faster alternative
4*64 2933 kit Pardon Our Interruption... $810.00 fastest 64 gb kit
PM983 1.92TB Pardon Our Interruption... $135.00 decently priced storage

Btw seriously 64 core modern cpu for 500 USD ? Both miracle and steal. Memory is expensive as always, but this is acceptable pricing for 512 GB ram total.

I don’t see you mention the DDR4 total per socket memory bandwidth speed, please be aware it seems to be a major factor for CPU run models. I run AI models on EPYC 8224P with 6 channels totaling 192 GB of DDR5. It works really well with such transfer speeds. I get for example

Qwen3:235Bn-q4_K_M at 5-6 token/s

GPT-oss-120Bn at 4 -5 token/s

and

unsloth/DeepSeek-V3.1-GGUF:TQ1_0 at 4 token/s

EPYC 8224P Click Connectivity to see Bandwidth
Per Socket Mem BW
230.4 GB/s (6 DDR5 channels at 4800 GT/s)

Your approach seems very cost efficient if you can be patient enough, but able to run large models for decent low investment.

if you already have an AM5 motherboard, putting in DDR5-6000 provides a theoretical peak memory bandwidth of approximately 96 GB/s.
I find the GPT-OSS-120 BN which only requires 61 GB of Ram to be really good at par with bigger models.
Top DDR5 8200 MT/s in two channels translates into around 120 GB/s of bandwidth

I dont do AI/LLMs currently or plan to selfhost them, so I dont prioritize memory bandwidth at all. Its a possibility in future, but slim one.

Might try it out of curiosity, but it this old hardware is expensive enough even with massive discount :slight_smile:.
Optimizing for LLM workload would multiply the price, even used. No go there.

To reiterate some points:

  • target socket / line and architecture is SP3 2nd gen epyc from ~~2020 , i.e zen 2 architecture
    • good enough for me, as it is almost on par perf. wise with work hardware
  • Platform is capable of running 8channels of DDR4 3200 reg. ECC with 1DPC, but I target slower memory for cost purposes
    • I also assume if I dont buy memory now as a kit, it will be much costlier in future. Hence initial target 8x64GB @ 2133. Overkill now, but who knows what will I do in a year or so.
      • Epyc 7742 theoretical maximum with 8C and 3200 config is around 200 GB/s, lets assume I get half of that
      • with current 16 core 7302 I got around 90 Gb/s througput wi 8x16GB 2133 DDR4 sticks.

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.