Return to Level1Techs.com

Watercooled E5-2680 V2 pfSense Build

I haven’t been using my workstation PC in a little while now so I decided to re-propose it as a pfSense box. Removed it from the Corsair 750D and waiting for my new Rosewill RSV-L4500 from Amazon. I will be cleaning and reusing the water cooling system within the Rosewill 4U case. I also am replacing my 7 year old Corsair AX650 power supply w/ an EVGA 650 G2 power supply.

I had a GTX 690 watercooled GPU inside the workstation but that is going up for sale on Ebay within the next coming weeks due to high power consumption that pfSense doesn’t require. I will be installing a simple EVGA GT 210 passive GPU instead. No noise and next to no power consumption.

Will be reusing the HyperX Predator PCI SSD along with 32 GB of 1866 MHz quad channel memory from G.Skill. I would have liked ECC memory but I should be OK.


Seems a bit overkill to be honest. I’m running this little thing and it does just fine:

1 Like

Yes, you are probably correct but I plan on running Snort along with a Squid proxy so maybe not so overkill. If you Instagram rpc_labs, his EPYC 7251 takes about 20 min to load the Snort rules. Not sure what network/CPU load would be during normal use but in any event.

I also plan on adding a wireless AP as well. Just purchased a Intel i350-T4V2 NIC of Ebay to support all of this. Still have to bang out the details regarding what rack mount gigabit switch (POE or not POE) is best/cost efficient that won’t give me any issues. Suggestions?

EVGA 650 G2 power supply and EVGA GT 210 video card arrived from Amazon. Once I receive the 4U case from Amazon, I will start laying out the water cooling config. Looking forward to it and not looking forward to it :slight_smile: OCD for cable and hose management.

The PCI layout to maximize the PCI bandwidth, specifically to the Intel i350-T4V2 NIC and HyperX Predator PCI SSD.

1 Like

Rosewill RSV-L4500 case came in. Installed motherboard, power supply, video card and SSD. Waiting for the i350-T4V2 NIC to come in from Ebay. Challenge -> Figuring out the best cooling loop config. Fun times ahead LOL

1 Like

I’m interested in how you get the loop in there. I have a couple of those cases, and I’ve always dreamed of doing a watercooled setup with the dual socket, multi-GPU system I have in one of them.

I did a little pre-fitting last night, don’t believe it will be too bad (ie. no custom metal work or mods).

You going to do a 360 rad on the middle fan bracket?

That’s what I was thinking of doing for my setup, but I’m a little worried it wouldn’t be enough for two 8 core xeons plus the GPUs. I have two GTX 770s in there now, but the dream is to have four 10-series cards. A single 360 rad might not be enough for all that.

Not a 360 but rather a 240 radiator (2 x 120 mm fans) that is 54 mm thick. I don’t believe a 360 radiator, regardless of thickness, would fit on the fan mounting bracket within the case due to the end tanks (ie. they hit the side of the case because there is not enough room).

My rule of thumb for sizing up your radiators is 120 mm (preferably 140 mm) for each component. Therefore, you would require 2 (CPU) + 4 (GPU). I believe you simply have too much horsepower (not a bad thing LOL) going on in that case, that you wouldn’t be able to fit this requirement without modification to the case. However, if you are not planning for the system to be under 100%, 24/7, a 360 “may” work. But, I plan for the worst case scenario: 100% load, 24/7 then add another 10%-50% capacity. Stay tuned, my next photo update will include the water cooling loop installed so I can better advise.

First modification to mount the EK-XRES D5 CSQ pump. Drilled two 8 mm holes into the case fan mount bracket to mount the EK-UNI D5 V2 holder. It just may be the only mod I will have to do. Fingers crossed.

IMG_20190115_1958286

Plumbing mounted. The ball valve used as a drain port is at the lowest position of the loop (when the case is put on it’s side). It will make draining much easier.

I almost think this build should run a Hypervisor with Docker allocations to everything and not simply be a pfsense box, with a SR-IOV card. You have a lot of cores, it can be better utilized with ESXi or a completely Docker based Linux distro.

You can run a few thin clients in addition to your setup, cause you have the hardware capable of doing a LOT more.

1 Like

What your proposing is actually out of my knowledge base. I wouldn’t have a clue where to begin or even if I require something like this. Stay tuned, more progress pics coming up tomorrow.

Noctua NF-P12 120 mm Redux fans came in for my radiator. Problem… the radiator screw holes do not line up with the 120 mm fan holes on the middle fan support bracket on the case. May have to manufacture a custom bracket.

I realized that my drain port is far too low, draining the system would be impossible LOL . A simple 90 degree fitting will solve this issue.

1 Like

I may have to re-think using a 240 radiator due to space and mounting constraints. Thinking of going with a 120 radiator but have to measure what thickness would fit ideally in the case.

Using the “one component, one 120 radiator” rule, do you believe I would have any disadvantage cooling both the CPU and motherboard VRM’s? I think I should be fine, what are your opinions?

Do I, due to space constraints:
A) Use a 30 mm radiator in push/pull or just pull
B) Use a 55 mm radiator in pull only (no room for push/pull)

Keep in mind if I only use one Noctua fan, I can only set it up as pull, as the case mounting bracket does not allow me to mount the radiator directly to it. I must mount the fan to the case bracket.

Well, this seems like the most logic setup. The 55 mm radiator is mounted to the chassis using one Noctua fan (NF-P12, 1700 RPM PWM) and is using one push/pull configuration on one side while the other side of the radiator has one Noctua (NF-P12, 1700 RPM PWM) in push. I cannot fit another fan behind this one but I will have more than enough cooling power and quite frankly, looks pretty good IMO.

The rear consists of two 80 mm Noctua (NF-R8, 1800 RPM PWM) fans.

Used 45 degree and 90 degree fittings on the end tanks.

I will be using EK-CryoFuel clear premix fluid (1L) as my fluid of choice. Used Mayhems red pastal fluid in this very machine/cooling loop for the past 4+ years, not a problem with it at all but I want clear going forward. In fact, the red color did not even stain the reservoir over this time. It is still crystal clear. Will be running Mayhems Blitz basic cleaning system through the loop for 12-24 hours prior to adding the EK-CryoFuel.

1 Like

Another update.

Watercooling loop plumbed, leaked tested and filled. Installed case and radiator fans, power supply 24 pin power and 8 pin CPU power along with some cable management. Intake fans and radiator fans all connect to a XSPC 8 way PWM splitter fan hub.

I really like the Performance PCS PCI fan bracket. I installed one 120 mm Noctua fan to cool the PCI SSD, Intel NIC and the fanless Geforce GT 210.

I will provide another update soon with my plan on controlling the fans.