Homelab Planning for Q1/Q2 2018 ~ this is gonna be fun

Hardware Build Logs
, ,
1

Hi all,

It has been well over a month now since I last vlogged and shared how much silicon was literally raining over at my place — which was the culmination of much planning & investing over the course of 4-5 months.

Quick re-cap -

Now…

What I’ve achieved so far with the homelab -

  • Installed the Dell P4317Q; insane productivity boost.

  • Added second Schiit stack, hooked up to the Threadripper workstation

    • This connects to the Dell over DP @ 60Hz

  • 7920x running Fedora 27, with 4+ VMs running. I’ve got a working Vault + Consul HA cluster working!

24
Screen Shot 2017-12-16 at 03.41.24.png1221x1035 121 KB

  • Windows 10 VM running great on the Threadripper workstation - to be used via the SPICE client (for the time being) to use XenServer’s Windows client software.

Next priorities

Within Q1 I would like to connect from my internal 24-port Netgear (unmanaged) GigE switch, the following topology -

  • Ingress (from the above mentioned switch) into the pfSense router

  • pfSense router to connect to a UniFi 8-port PoE managed switch

  • Install the 7920X box as a dedicated XenServer on a VLAN off the UniFi 8-port switch.

  • Install remaining RAM to get this to 32GB.

What’s evident is that I need a ‘sandbox’ Fedora box for playing with virsh; it is super easy to get things up and running; backup the .qcow2 images and associated XML data to move VMs around as needed. New build for 2018 =

  • Intel i9-7940X (14C / 18 Threads) ~ $1,400/- CPU. I don’t mind the premium over Threadripper if it means less headaches and I have the option of running BSD (if needed).

  • Either another Strix-E X299 or I’ll snag a Rampage board.

  • Buy 32GB RAM.

Upgrades -

  • Threadripper workstation needs another 32GB of Dominator Plats.

    • I’m already exhausting my 32GB RAM when running the Win 10 VM. Without that, I’ve used 70% of RAM as I have a couple VMs running on this as well.

  • Buying an EdgeSwitch ER-16-X for 10G over SFP.

2 Likes
2

I also have to -

  • Sell the Gigabyte Aorus Gaming 7 X399 Theadripper board

  • Return a New Zenith Extreme X399 board to Amazon

  • Return a (new) Damaged on arrival Corsair K95 CherryMX Brown keyboard back to Amazon.

  • Warranty RMA a Corsair H115i (for repair)

  • Warranty RMA a dead/dying Corsair AX860i (from Nov 2016…)

3

So. Much. Hardware. Sounds interesting, so I’ll be keeping an eye on this thread.

1 Like
4

This is cool. My homelab plans are having to be scrapped because I have to move out of my apartment when this lease is up.

5

Sorry to hear that @Dynamic_Gravity

Couple things have gone right and wrong in the last week or so… TL;DR I found my pfSense build from August-ish wouldn’t boot up. Ended up swapping out the Asus TUF Z270 Mark I // 7700K combo for my older (from Nov 2016) Asus Z170 Maximum Formula VIII // 6700K — both very overkill for a pfsense box, but hey, the 6700K boots/runs fine.

Since my return deadline to Amazon is looming up (end of the month), I’ll be shipping back -

  • Corsair AX760, ordered this to try and isolate a dead-ish PSU on the pfsense; I really need to keep a spare PSU on hand. Will be doing so, but I’m going to keep something in the 860W range as surplus.

  • Asus Zenith Extreme (the second board); still sealed, going back like mint to Amazon.

  • Asus X299 Strix-E that’s going back too.

Just placed an order for a ROG Rampage VI Extreme cause #yolo I get the 10G support, but most importantly better OCing, DIMM.2.

So what’s changed? the 7920X is going to be a hybrid-box. Virtualisation server - yes, but also a really HEDT. GPU pass through - very likely. Things I’ll need to plan and add to make this happen

  • Another Corsair AX1200i PSU. I just can’t complain about them (so far).
  • Ideally 64GB 3200/3600MT/s (as per der8auer, over 3600MT has minimal gains).
  • Additional 960 Pro Nvme drive.

Why send the Strix back - bit of a mixed bag, I’ve had more RAM trouble with it and the same RAM has been stable on the z170 Maximus board in the past.

Arriving this week

  • Ubiquiti UniFi managed 8-port (lv2) PoE 150W GigE Switch (has dual SFP uplinks if one needs that sort of thing). The setup will be pfSense router -> UnFi Switch -> VLAN magic out the end.

What have I achieved?

In the same span of time, with the pfSense router + 7920X I was able to setup a VPN connection directly into an AWS VPC. That’s 2x IPSec tunnels. And I used terraform to aid in the whole ‘infrastructure’ as code aspect.

Advantage - took me a while to get it working perfectly, fine tuned it, and now I can go into any ‘fresh’ AWS account and have everything needed/attached and configured to bring up a working VPN connection in under 5 mins (longest of that is AWS needing time to provision the VPN connection) — or to be precise automate the creation process of as many VPCs tied to VPNs within a ‘global’ AWS account.

I also ended up setting up my primary EdgeRouter as a DMZ Host under my ISP provided crap-router (ZTE branded thing). There was a fair bit of NAT/firewall magic needed to ensure the IPSec IKE/IP-50 (ESP) traffic was being passed to the pfSense box correctly.

Was it painful? Yes. But oh so worth it. Costs ~$36/month for a single VPC>VPN connection.

Use case for AWS VPN

Suppose I hire a dev to work on a special project, I can set him up isolated in his own VPC sandbox, with traffic off a specific VLAN. He can be totally managed by his IAM profile, Cloudwatch/logs. Any iffy business, I’ll be able to stay ahead of it. Mind you - I prefer to work with those of the hacker mindset, but I also have to take a pragmatic approach to keeping a watchful eye… :wink:

What’s nice is Suricata can also be set to run on the VLAN assigned for further introspection.

CC @SgtAwesomesauce cause I don’t think too many here would care for the AWS related babble…

2 Likes