Recommendations for build server

Hi there!

My team is considering buying a server to be used primarily for compiling various Yocto-based projects. I’ve been “asked” to recommend the best specifications, but this is a little outside my area of expertise.

The server will be used primarily as a build server, with at least 10 users building multiple targets at the same time. Based on our current usage, I estimate that we will need around 6TB of storage.

I have a few specific questions to start looking at my options. My budget is around $18k:

  • Should we always opt for dual CPU configurations over a single CPU with the same total core count?
  • Should we go for NVMe storage, or would SATA SSDs suffice?
  • Is it better to invest in a higher-end CPU rather than faster storage?
  • As I have multiple users building projects, is it a bad idea to aggregate all the storage with LVM or similar rather than mounting all the drives separately?
  • Is VROC worth considering in this context, even if I go with non-NVMe storage (if I go with an Intel CPU)?

Thanks in advance for your insights!

1 Like

If I understand your workload, then what you need is a server that can handle heavy compile workloads, which are heavily CPU dependent. That being said any server CPU is capable to handling that workload the question really is where you want to put your money on this one. It is pretty easy to drop 5-6K on the processor, MB, and memory alone. Then there are the storage requirements, you say 6TB, but remember to double that for redundancy. So 12TB of storage, but even that is fairly light and can be accomplished fairly easily with SSDs. So were it me setting the specs I do something like this:

CPU: Any 32 plus core Xeon/Epyc CPU
Memory: 1-2TB
Boot drives: redundant NVMe
Storage drives: Redundant 10TB RAID(5 or 6) arrays
Network: Redundent 25G ports

3 Likes

You’d need more than 128Gb of RAM which makes your choices limited to enterprise, a “normal” NVME drive will do fine as far as I/O performance goes but you probably want redudancy as TeamTux mentioned, a single build can eat up 64Gb so if you’re going to do multiple builds * 10 you’re looking at a very beefy box so what TeamTux suggests seems to be pretty much in line with that I’ve heard about Yocto. That being said, given what a Threadripper or Xeon CPU costs with MB etc you might want to consider building a small “pool” of buildboxes. If you’re targeting ARM you might even want to look at such a box Ampere Altra/Neoverse) which can be equipped with a ton of cores.

Amount of cores doesn’t scale 1:1, for example a 7960X isn’t twice as fast as a 7900 but is ~4X times as expensive. Granted you need to account costs for motherboard, ram, storage (unless you use ISCSI or something) you also get a bit of redudancy.

2 Likes

For example this might be an option, no idea how good of deal that is price-wise though.
You might want to do a bit of research as always to avoid getting something with hardware quirks etc.

1 Like

I take there is a server-room, air-con, UPS, etc. already installed/available to use?

Just ran rough specs for this through Supermicro’s configurator on the AS-2125HS-TNR, comes out at 18k-ish (well, 20,500€)

Part Name Count Note
Platform Supermicro AS-2125HS-TNR 1
CPU AMD 9374F 2 32 core
RAM 1,152GB 12 96GB sticks, may be suboptimal
Boot disk Any 1
Storage D7-P5520 (3.84TB) 6 15-ish TB useable
Network Intel E810 25G 2

Other then being a bit below where I would want the storage to be it does check all the boxes.

Is there network storage somewhere you can make available on the machine?

Or add a SAN later on.