Dual xeon build questions

Hardware Build a PC
1

I like to shop around on ebay and look at the current prices for used parts, specifically trying to use older server hardware, since there's usually pretty good deals. Lately I've been toying with the idea of doing a dual xeon build, but using the extra performance to run more than just windows/linux using virtual machines.
Before I get into the software and theory aspect, here's the hardware list.
CPUs - intel xeon e5 2670s
Cooler - hyper 212
Mobo - AsRock EP2C602-4L/D16
RAM - 64gb ECC ddr3
HDDs - WD black 1 tb, hitachi 2tb, 4x 4tb hitachi nas
SSD - 512gb crucial mx100
GPU - 295x2
PCIe - asus xonar soundcard
PCIe - TV Tuner
PSU - seasonic x-1250
Case - fractal design r5

My reasoning:
CPUs - cheap as hell for the performance I'm going to get. Around the same performance as my 8350 for a single chip, let alone with 2, and for the price you cant beat it.
Cooler - lets be honest, this is the budget choice, no two ways around it
mobo - this is the main issue here, I love the multiple nics built in and the 3 pcie x16 slots, but it is hard to find this motherboard, but again for the value especially open box it is crazy hard to beat.
RAM - ram has really come down in price and I can get an 8x8gb set on ebay for around 100-130 with ecc, so why not, it's also always good to have extra ram for VMs
HDDs - Bringing everything but the hitachi 2tb drive, that would be an addition, the 4tb drives would be pulled from my current nas
SSD - would be bringing this over
PCIe - I would need a soundcard, since most server motherboards lack this feature, and would also bring my tv tuner over, would need a lsi raid card for the nas, and usb card for ports for general use
PSU - bringing this over, it's a super solid power supply though so even better
case - bringing this over, love the case

Problems with hardware:
I cant find a good motherboard. I'm looking for dual socket 2011, at least 2 pcie x 16 slots, and at minimum 2 nics. Any suggestions? as close to or under 300 as possible

The case does not support most of the motherboards I'm looking at because of the slope in the motherboard tray for cable routing. By cutting this out and putting in metal braces, I would hope to compensate for this with appropriate mounting points. Everything would be nice and cramped in there, especially with having to have the HDD cage in the front, but everything should fit based on my measurements

Problems with software:
Specifically, there are issues with running freeNAS (my preferred NAS solution) inside of a virtual machine. It can be done but it has to be done right, which is where the lsi card comes in. This would let mo connect all of the drive and just pass the card and controller through, at least as I understand it, solving many of the problems. Beyond that, it appears that if done correctly there are few drawbacks to running FreeNAS in a VM, though please correct me if I'm wrong.

I've not done too much with VMs before, and I've not had enough time to research, although I see a lot of talk about esxi and xen in terms of freeNAS in a VM. Will it be possible to actually run the nas and a windows VVM alongside each other, and still pass my hardware through to each? TV tuner/usb/soundcrd/gpu >> Windows, lsi card >> freeNAS

Can you pass a hardware device through to more than one VM at a time? for instance, can I pass the gpu through to a windws vm and linux at the same time? If not, would the 295x2 allow for this since it it technically 2 290x?

Is there anything I'm missing? this seems like a fun project to learn about virtualization while also letting me upgrade and make my rig into something unique. I game, but I also do some productivity work, where an upgrade in the cpu department would be appreciated. More than anything though I think this just sounds like a fun way to try to get killer performance for a not so killer price. Being able to have a windows gaming rig, linux workstation, and nas all in one rig running at the same time with no compromises sounds pretty sweet too.
At this point all this is is a hobby project idea, but before going further I figured I could ask here and see if there are any specific problems with what I have in mind.

tl;dr
I want to use older server hardware to run windows, linux, and freeNAS at the same time, will this work? Specifically, will freeNAS still work with these if configured properly with pass through? Can I pass a device through to more than one VM at a time? Is the upgrade to 2 e5 2670 a worthy upgrade over a 8350, given that the new rig would be for everything NAS included, as opposed to just a desktop?

2

At this point dual xeon boards are super dead. The best thing you could do in my opinion is get the newest possible Mac Pro that was a tower (350 bucks max) crank that shit to 11 (procs are cheap, and I think the last ones before the garbage cans had DDR3 but someone check me on that as I have more BG on the PPC ones), plus good airflow and you can put a 380 in it with little problem. Latest OSX plus more drives and multiboot you'll have a boss as hell box all around. I want to do this and use it for Wine testing in OSX and make it a render box.

3

I can only find stuff with those requirements for $450 at the minimum, your going to be stuck trying to find something at $300.

No and no. The 295x2 really doesn't support being broken out like that and being used in 2 different VM's at the same time as two 290x's.

I don't really have the expertise to answer more questions more specifically about virtualization specifically, but I can say that for really multithreaded applications dual 2670's will destroy a single 8350.

4

@FaunCB I would, but the idea here is to kinda consolidate everything I already have, and to do it by reusing as much of what I have as possible.

@thecaveman that's what I was afraid of. I might keep looking though. Thanks for the information about the video card too, that should be helpful no matter what, as well as the info about VM passthrough

1 Like
5

another possibility is doing the mac idea, only tossing mac os and using windows or linux.

6

seeing as you mentioned this too, what is it about the macs that make them so good for this kind of rebuild? Why haven't I heard any of this before? Why is this preferred to something like dual xeons? (as Aremis mentioned)

7

well it's a dual cpu motherboard with a nice case, nice air flow, psu, ram, 2x cpus, and a random gpu for a few hundred tops. once you replace the cpus and throw in some server grade ecc ram you have a 12 or more core pc with 64gigs of ecc for under 500. the cpu's and the motherboard new alone will cost you around 1k or more depending on brand ect... cost wise this is pretty nice since a lot of non computer literate people who buy macs will just buy a new one instead of upgrading as apple actively discourages upgrades, and you can get older systems cheap then upgrade yourself.

tl:dr cost/performance it's pretty nice for things that need large amounts of cpu cores and ram.

8

Don't fall into the trap of believing you need lots of cpu cores to run lots of Virtual Machines. VMs usually become bottlenecked by storage IO and lack of RAM before CPU resource becomes tight.

That being the case you might want to question your current mobo/cpu choice. A skylake i7 or Xeon could well be a good choice given they support all new IO interfaces and 64GB.

9

only one thing to say regarding this.if you're going with the 212, make sure that the mobo is square ilm. because most 2011 xeon boards are narrow ilm. also server os shouldn't be running on a vm. not a good recipe. does freenas run vms natively? i'm sure even if its not native. someones gotten around to making a mod for it by now. i know nas4free does virtualbox natively.