Hi everyone,
I am theory crafting proxmox server build around 64c epyc cpu from 7002 series - 7742 specifically.
My usecase is personal and professional learning/proofing platform on a budget. I am getting into a lot of automation lately and having virtualization host where I can run simultaneously:
- redundant kubernetes cluster
- redundant ceph cluster
- redundant reverse proxy
- redundant database servers
- windows AD or equivalent
- windows DNS
- windows ADCA for local cert issuance
and additional services as needed without having to leave single machine would be very useful. 64 core cpu might overkill, but they got very cheap lately. Memory is decently priced too, so I can get 4GB/core (8x32) or even 8GB/core (8x64) without braking a bank.
Normally this would be completely out of my budget, but I already own used supermicro H111SSL-i with 16 core epyc 7302 and 8x16GB 2133 ecc ram as a kit.
Replacing cpu and memory to upscale it is affordable enough. For extra ~~1000 USD I could build 64core machine with 512 GB ram and thats very tempting. Or small dick syndrome, IT edition.
Does anyone have experience with this setup? Any surprising or devious pain points?
Points of interest:
- cheap memory is slower than officially supported optimum, 2133 vs 3200. Is it worth paying extra and what is power consumption tradeoff here? I doubt I would benefit from increased throughput, but better know it before ordering anything.
- 64 core cpu might stress board power electronics so that extra cooling might be necessary, is instability common?
- 7003 series 64 core cpu droped in price as well, but they are more expensive and require new pcie 4 capable motherboards. Those are extremely expensive, even more so when I bought the original kit. (supermicro H12 series)
- I do not need pcie 4, and given the price is not worth extra investment even for nice to have futureproofing
Supplemental data:
- idle power draw is irrelevant, since I intend to power on system remotely only during active work. It will be powered off otherwise. Its purely intended for workload that I build, test, break and then cleanup. Any critical services would be run on separate power efficient proxmox host (ucff ryzen hp elitedesk ftw)
- original purchase was meant to replace old xeon e3 1225 v3 nas system, but idle power draw of 7002 series was bad enough to nix it. Nas runs 24/7, so it would get expensive really quickly
- all part will be second hand and imported from china via ebay, I have good experience with specific seller. Financially absolute no brainer, there is no sane alternative to it.
- I would prefer getting used ThinkStation P620 with Threadripper PRO 3995WX , but those things have no secondary or sanely priced market.
Prices I am basing this on:
| Part | Ebay link | Purchase price | Current price | notes |
|---|---|---|---|---|
| 7302+H111SSL-i + 8x16GB 2133 ECC | Pardon Our Interruption... | $577.80 | $1,025.24 | already owned |
| Epyc 7742 unlocked | Pardon Our Interruption... | $499.00 | cheapest non oem article | |
| 8x64 2133 kit | Pardon Our Interruption... | $656.00 | slow but massive, cost efective | |
| 8x32 2133 kit | Pardon Our Interruption... | $464.00 | ||
| 8*32 2666 kit | Pardon Our Interruption... | $659.60 | faster alternative | |
| 4*64 2933 kit | Pardon Our Interruption... | $810.00 | fastest 64 gb kit | |
| PM983 1.92TB | Pardon Our Interruption... | $135.00 | decently priced storage |
Btw seriously 64 core modern cpu for 500 USD ? Both miracle and steal. Memory is expensive as always, but this is acceptable pricing for 512 GB ram total.