Quiet Dual Xeon Workstation Build for Multi-core Dev and Audio

Hardware Build a PC
1

Hi Everyone,

I haven't built a machine for a long time. I'm just starting to plan this one. After watching the Tek Syndicate Dual Xeon Workstation build videos I would like to ask your advice.

[EDIT: I've updated this post, and added a parts list from pcpartpicker]

My goal: create a quiet, balanced, economical workstation, given the constraint of using two high-core-count E5-26xx v3 CPUs.

My requirements:

  • Dual processor, LGA2011-3. I'm specifically building this machine to research high-core-count CPU-bound algorithms, so high core count is a must (roughly, at least 10 per socket).
  • Use 1: Research and Development workstation (Visual Studio C++, parallel builds)
  • Use 2: Audio workstation. Must be quiet. I'll run with my MOTU USB2 interface. Maybe RME PCI later.

Notes: No gaming. No 3D. No co-processor requirements. No plans for overclocking. Expect workloads to be a mixture of CPU-bound/memory-bound/IO-bound. Pretty-much decided on air cooling. OS will be Windows 10, maybe Linux and other OS under VMWare. Might upgrade CPUs or graphics later. Location: Australia.

I've listed the parts that I'm considering below, along with some questions. I'm really interested in suggestions for alternate components that might better suit my needs.

http://pcpartpicker.com/p/g8qvgs
(NOTE: I have specified 2x E5-2698 v3, PC part picker doesn't list E5 2675 v3.)

Motherboard:

Z10PE-D16 WS (EEB)

Chosen to give more flexibility with RAM upgrades than the 8 DIMM models. Also considered ASUS Z10PA-D8 WS (ATX), ASUS Z10PE-D8 WS(EEB). Should I consider anything else?

CPUs:

2x E5 2675 v3 (OEM, 16 core 1.8 Ghz)
or
2x E5-2683 v3 (14 core 2 Ghz)

These are relatively cheap second-hand compared to higher clocked, lower core-count models. The 2675 is cheapest, that's good, given that my goal is high core count.

I'm wondering whether memory bandwidth will be a problem with so many cores. In particular for Visual Studio parallel builds. What do you think?

RAM:

32 GB: Crucial (4GBx8) DDR4 2133 MT/s (PC4-2133) CL15 SR x8 ECC Registered

4GB DIMMs chosen to use all 4 RAM channels for both CPUs.

I am wondering whether I'll have issues with RAM physical clearance and CPU coolers.

CPU Coolers:

2x Noctua NH-U12S

Seems like the main quiet and cool option. Are there others that will fit on a two-socket motherboard? (Noctua NH-L12? Noctua NH-D15?)

Case:

Fractal Design XL R2

Has been used successfully by others with Z10PE-D16 WS and has received positive quietness reviews. Fractal Design define R4 was recommended by 2013 video. I'm open to other suggestions. Highest priority will be quietness and ease of build. Obviously I need something big enough for the dual processor setup (and maybe the EEB motherboard).

PSU:

Seasonic XP-1050 Platinum 1050W [UPDATED, was Corsair AX860]

Chosen for quietness and SeaSonic manufacture.

Also considered Corsair AX1200, AX1200i. Requirements: Quiet. Enough headroom to handle an upgrade to 160W CPUs and 1 high-end graphics card. Compatible with Haswell C6/C7 low-power states.

Graphics Card:

ASUS Radeon R5 230 Low Profile 2GB Video Card

Fanless. Will upgrade later if I need to. Will need to research driver DPC latency for audio applications. I'm open to suggestions.

Storage:

Undecided, but considering SAMSUNG SSD 850 PRO 2.5" SATA III 512.

UPDATE: Also considering PCI NVMe: Samsung 950 Pro or Intel 750.

I'd love to hear your comments and advice.

Thank You.

2

Alright well you have fun with that.

I don't know if there's other older Xeon 16 core options, but you could save like 1.5k going with a 4 16 core opteron build for the same amount of cores/threads, but naturally there wouldn't be as much CPU performance.

$800 for a 16 core Opteron, better off going intel probably if you have the money
http://www.newegg.com/Product/Product.aspx?Item=N82E16819113320&cm_re=Opteron-_-19-113-320-_-Product

quad socket board, only lists 12 cores there, don't even know if it would work actually, but info is info I guess
http://www.superbiiz.com/detail.php?name=MB-H8QG6F&c=FR&pid=e272a92379836841df6dda81ec3938d73ac91c89c5ab754944d2ec8739895007&gclid=CO218ZyOiMsCFQyNaQodfwALQw

probably worth throwing in a PCI-e SSD to remove bottlenecks there

There's also the cluster option if you're into that, you could go with a bunch of 8320e machines and have way more cores overall

Also go for a higher end PSU man, RM 1000i
http://pcpartpicker.com/part/corsair-power-supply-cp9020084na

Are you sure you don't want a GPU with a lot of display ports for many monitors?

3

Thanks Streetguru,

I have my heart set on Xeon. I've gone ahead and bought two E5 2675 v3 (total AUD 1369).

I hadn't considered PCI-e SSD, but that seems like a good idea for use as a working drive, if not for boot. Can you recommend anything? I started looking into Samsung 950 PRO M.2, but the Z10PE-D16 WS motherboard seems to only have PCIe x2 on the M.2 slot, so I'd need to use an adapter into a PCI slot -- I guess that could work. My current plan is to boot from SATA SSD (Samsung 850 PRO). The NVMe boot story for that motherboard seems a bit sketchy. No official Windows 10 support either. [EDIT: latest ASUS drivers apparently support Windows 10 and NVMe boot.]

After further research I'm tending towards Seasonic PSU for quietness and quality. $30 extra for the 1050W is an easy choice, but I feel like 860W is already overkill given the current spec.

Seasonic XP-860 Platinum 860W Power Supply V2 ($325 AU)
Seasonic XP-1050 Platinum 1050W Power Supply ($355 AU)

It's possible that I'll add a GPU later. For now I'm happy with my single monitor setup. And I'm reluctant to buy something with a fan on it.

4

Probably the intel 750 stuff maybe, nvme drives seem to need fairly substantial heatsinks, M.2 might not be the best move

I think most of corsair's stuff just rebranded seasonic stuff

oh right, Australia. go for that then ya.

5

Yes, it looks like M.2 950 Pro can get hot in heavy benchmark workloads. Not sure what real-world performance is like. A heat sink adapter exists, which supposedly resolves the heat issue. It's not cheap: Angelbird Wings PX1 ($110)

All of the options are beyond what I was going to spend on SSD, but I will consider them.

  • Intel 750 Series 400GB PCIe ($558 AU)
  • Samsung 950 Pro M.2 512GB PCIE Internal SSD + cheap PCIe 4x adapter ($418 + $22 = $440 AU)
  • Samsung 950 Pro M.2 512GB PCIE Internal SSD + PX1 ($418 + $110 = $528 AU)
6

You could just like buy a small heatsink and slap it on with a thermal pad, and I don't know how well the cheap adapter would work out, best to do things directly if at all possible

something like this I think

7

Corsair PSUs rebrand Seasonic, Flextronics and Channel Well. It varies by model. There's a good list here:
http://whirlpool.net.au/wiki/psu_manufacturers

8

Some kind of M.2 to PCIe x4 adapter is required because the M.2 socket on that motherboard only has PCIe x2.

But yeah, cheap adapters.

9

Well I dunno about AUS, but here's a decent adapter that's x4

http://www.newegg.com/Product/Product.aspx?Item=12K-017B-00001

and I guess the Intel NVME drives have better random read/write compared to the 950 drives, might be worth going intel just for simplicity's sake, you're already spending a metric shit ton on the PC itself, why worry about $100 on the drives?

750
http://www.newegg.com/Product/Product.aspx?Item=N82E16820167359

950
http://www.newegg.com/Product/Product.aspx?Item=N82E16820147467

10

Why dual xeons? If you're willing to kick out as much cash as needed, theres newer xeons that do 4 times, probably, as much work as the 2 older ones. There are boards for it but I am curious as to why.

11

Aremis, good question.

Mostly for flexibility. The 2 socket thing is an experiment. I'm looking to scale some algorithms that I'm developing. Being able to evaluate these algorithms on multi-socket setups is helpful. It will give me first-hand knowledge that is valuable for my product and my work. The conclusion may be that multi-socket isn't worth it (for my algorithms and/or for my users.) I'm balancing that risk with the hope that I can scale my Visual Studio parallel builds over both sockets.

I picked up 2x E5-2675 v3 (OEM, 16 core 1.8 Ghz) for 942 USD. That seemed like a good way to get into dual socket LGA2011-3 "for cheap". Something like a single E5-2699v4 22 core is way out of reach (e.g. even an engineering sample is on eBay right now for USD 2100).

I'm over my budget now. If the budget really was unbounded, and I planned to build another complete system in 1-3 years, then a single-socket X99 solution with a faster CPU may well give better raw performance. But it would be near the top of the "upgrade window." I like the idea of having an upgrade path. I figure that if the E5-2675 v3 setup isn't optimal, in a year or two I can swap in faster, maybe lower core count CPUs -- and they'll be cheaper on the second hand market than they are right now. Worst case I could even switch to a single CPU -- the Z10PE-D16 has enough DIMM sockets to reuse all my RAM on a single CPU.

1 Like
12

I can see what you're talking about. I've never really had the high end before when it was actually high end. I'm using a phenom x4 and plan to get an x6 and a 380 and an am3+ board for later upgrades then to am4. I like the idea of a build and upgrade path too I've just never used multi-cpu setups XD

13

Aremis, I forgot to ask you something. My CPU choice is:

2x E5-2675 v3 (OEM, 16 core 1.8 Ghz, Haswell-EP)

Is there really a single CPU that would do 4 times as much work as this?

14

Really just a guess but probably not. As I said, and I didn't know your situation, never used a dual CPU setup and they've kinda gotten kicked aside nowadays. For dev work you're definately good. Might do better with newer chips but the question of finding a board is one to be answered. Let me know how it comes out though!

15

Thanks. That board supports Broadwell-EP. Support was added in a firmware update in late 2015.

1 Like
16

AM4 is coming next month apparently, might as well jump on then if it's priced decently, the desktop carrizo chips will be out on it.

17

I thought that stuff was slotted for Q4?

Sorry for derailing OP

18

That's zen CPUs, the motherboards themselves are supposed to be out in march

http://wccftech.com/amd-am4-motherboards-launch-q2-2016-for-bristol-ridge-apus/

19

For reference, here are my known options for M.2. PCIe adapter with heatsink:

https://www.ramcity.com.au/buy/bplus-m.2-pcie-ssd-to-pcie-2.0-x4-adapter/M2P4A
https://www.ramcity.com.au/buy/angelbird-px1-pcie-x4-adapter-for-pcie-m.2-s/WIPX1-FWKF

These guys recommend a heatsink with the PRO 950, as it thermal throttles during sustained file copying:

20

Going to want the x4 adapter naturally, and ya heatsink's good, but I guess it depends on weather you want general read/write speed or random speed vs the 750 SSDs.

If system stability isn't a major focus though you could always just RAID 0 sata SSDs to get 1GB per second basically. just have a back up of everything that's regularly updated.