Moving: Setting up a NAS, pfSense, and VPN

Hardware Hub Networking
1

Hey!

So, I stumbled here through the jungle that is Youtube. I'm in a awkward transitional period with my hardware and moving into a new place this coming month will be the catalyst for hopefully setting up a proper NAS and router-based VPN.

Currently, the gear I have is:

I'm hoping to connect myself and my two roommates to the physical LAN, total, we'll have 5-6 computers on Ethernet and we'll have several devices on wireless. (Smartphones, laptops...)

My VPN provider is Private Internet Access, and my ISP will be Cox. I want as much of our traffic to go through PIA as possible, making exceptions for gaming services and trying to work around any complications like if Netflix gives us a hard time.

I originally bought this Supermicro board to set up a NAS, I'd like to have lots of storage space for random data and also for 1080p @ 60fps (or 120fps) footage, and besides that, it would be nice to actually start backing up Windows installations and things like that. If it is possible, it'd be great to have network-based "scratch space" to record to and edit from, but I don't know if that's worth a large premium. Said 'scratch space' would need to be able to fit hours of footage, probably 8 or more at a time?

My intution tells me that setting up a hypervisor on the VPN would be able to take care of everything, but I'm very newbie-level when it comes to Linux. I tried setting up oVirt at one point and I couldn't get it up and running.

So, I have a couple of questions:

My ASUS router is not powerful enough to handle a (slower) connection going through my VPN, and it slows down to less than the ~7Mb/s my desktop's NIC slows down to when using the VPN. Is there any chance the Supermicro board can handle a 300Mb/s Down, 30Mb/s up connection without losing a ton of speed? Can it do that inside of a virtual machine?

I've heard that certain NAS setups don't play nicely inside of VMs, is that true? I know that some NAS services (like Unraid) support virtualization, would any of those work as a hypervisor for pfSense?

Having my router inside of a virtual machine on a bare metal NAS gives me a bit of a headache, I know on paper it should work but it seems like there could be weird situations where the pfSense VM hangs up or crashes and suddenly my entire network doesn't work... I guess that's about as likely as a consumer-grade router needing to be reset?

Do you folks have any suggestions for me? Also, should I keep using the ASUS router as my wireless access point, or should I invest in a nicer one? It's worked pretty well for me so far, but I've never had so many devices connected to it all the time, I've been living by myself since I got it.

If I need to make any hardware acquisitions or upgrades, I'd like to focus on value rather than shiny awesome stuff.

Thanks!

1 Like
2

Phew lot of questions so I'll just post what I'll be doing since it's very similar. What I'm going to be doing is setting up ESXI 6.5 and visualizing both my PfSense install as well as an installation of FreeNas.

I will be using very comparable hardware actually. I will be using a Xeon E3 1230 v1, 16gb of DDR3 Unbuffered ECC, a LSI 9211-8i HBA, an Intel Pro 1000PT 2 port, and an Intel server motherboard. The Xeon 1230 v1 is actually very similar in terms of cpu performance to your Avoton as well.

You could try doing something similar to my setup, as I will detail below, as I already had it "running" before the FX 8350 I was using straight died.

I will be setting up 2 vms.

Vm1 - PfSense
PfSense doesn't need a lot of ram to perform well, with VPN traffic or anything else really. I will be able to get away with 1 or 2 cores available to it as well as 512MB or 1GB of ram. It is extremely lightweight and doesn't need much. Looking at my pretty bare PfSense machine right now, its using 12% of the 2GB of ram it has installed, and the cpu never peaks over 10% utilization with an Celeron J1900 cpu. This VM will be pretty easy to setup, just assign it resources and make sure its setup to auto-start. I will also be passing through the dual port Intel Nic to this machine, as I want dedicated networking ports for the WAN and Lan connection. You can get away with allotting it one Nic Port and having the Lan traffic go out over the other machine ports, but I prefer to have it given its own hardware.

VM2 - FreeNas
The key to FreeNas is that it needs a lot of ram, and it needs to have direct access to the hard drives. Simply passing through the hard drives did not seem good enough to me, so what I've opted to do is pass-through the LSI HBA card to it so that it can have direct access that way. I'll be giving FreeNas access to 4 or 6 threads, as well as 8-12GB of ram since it is very ram intensive. It doesn't really need a dedicated Nic in my setup because the other nics on my motherboard will be able to more than handle the traffic without issue.

If you want to try this, you'll need an Install of ESXI free which can be done through this method:

It's free and for a one machine install, its great. Most don't think of Esxi as an option for a home server, but actually the free install is very powerful and can be used to do a lot.

2 Likes
3

Are there any kinks to expect when setting up Esxi? What about pfSense?

Is it a good idea to set the machine up at my current place behind my (AT&T) Modem and my ASUS Router, then just transfer it over to the new place and transition to a new network based around the new router?

4

Not really, ESXI is a very simple install. You can install it to a Flash drive. Personally, I've got a Sandisk 64GB flash drive that actually reads and writes about as fast as a spinning hard drive, so I'll be installing it to that.

Pfsense is again a very simple install. I'd just make sure that you know which nic or nics you pass through to it so your not plugging in your connection from your modem to anything but the right port.

What I'm going to do is install ESXI, then setup the PfSense VM with all the PfSense NICs unconnected, assign all the network cards as they should be, then transfer over my WAN and LAN connections once everything looks good. For installing PfSense under ESXI, you can follow this guide:

Personally, I would replace your Asus router at your current place with the ESXI and PfSense combo to make sure everything works before you move to the new place.

1 Like
5

Wow, thanks for that video. It made things look really, really simple. I think I have the skillset to tackle this!

6

Great! If you have any questions I can probably help since I've set up one of these boxes before and have another motherboard on the way to re-do it myself again :smiley:

7

Hey, I just installed ESXi following the guide you linked and it seemed like everything worked out. I've booted into the hypervisor and here's what I'm looking at:

VMware ESXi 6.5.0 (VMKernel Release Build 4887370)
Supermicro A1SAi
Intel(R) Atom(TM) CPU C2758 @ 2.4GHz
8GiB Memory
Download Tools to manage this host from:
hhtp://169.254.130.15/ (Waiting for DHCP...)
http://[(ipv6 address)]/ (STATIC)
Warning: DHCP lookup failed. You may be unable toa ccess this system until you customize its network configuration.
<F2> Customize System/View Logs <F12> Shut Down/Restart

So, I have a keyboard plugged in and F12 responds no problem. I can log in and ask the computer to shut down, but F2 does nothing. Sometimes the screen blinks when I press it, but it doesn't actually seem to activate or change anything.

If I had to guess, it's selecting the wrong NIC? Each port on the board, when plugged in, has an orange/red light instead of a green one, the activity light blinks but no address is ever grabbed.

Edit: As far as I can tell, every other key on the keyboard works just fine. F4 makes the screen black and white instead of colored, some of the other keys do stuff. F2 does nothing though.

Edit 2: My router sees the device on the LAN and even assigned it an address (2.2.2.76), I can query it for info and see that it is in fact a Supermicro device, but it doesn't respond when I try accessing the address from my browser. ESXi has not recognized that it's been assigned an address.

8

Very odd. My ESXI install does a similar thing where it doesn't really know which nic to use for management, but its very odd that yours would do that. You can tell it which nic to use if your F2 was working, and its very odd that its not. I would try another keyboard, or a different usb port.

9

I've tried several USB ports, I'm about to go run some errands and get lunch and I'm going to grab a keyboard from work while I'm out.

1 Like
10

Ok, it turns out my keyboard was the issue. Weird, but I've seen similar stuff with recent Chinese keyboards, trying to use them on Android devices. (Different keyboards, but they were from China and so is this one I'm trying to use right now...)

This Logitech K120 I picked up from work got the job done and I was able to switch my Nic, log in via the web browser and--fingers crossed--it should be smooth sailing from here.

Here's a weird question:

Does ESXi support PCI Bifurcation(Sp?)? Any idea where I could ask about bifurcation with my board, a network card, and a hard drive controller card?

11

I have never even heard of this to be 100% honest. My recommendation would be to go ask on the VMWare forums. These avoton boards have been quite popular for home servers, so I'm sure there are others there that will know better if it supports that feature.

1 Like
12

Hey, more pertinent question:

How do I get the client software that My PlayHouse is using? I keep digging for it and searching but can't find anything. I'm trying to follow along with the guide, but the web interface is pretty different from the one he's using (and his looks better).

13

The software he's using is called vCenter if I remember right. A vCenter version 6.5 was never released with ESXI 6.5, so the closest version is vCenter 6.0. Sadly, if you use 6.0 with a 6.5 server, you can't do everything 6.5 has to offer since vCenter isn't updated to handle it. It makes you use version 12 VM's instead of version 13 VM's, and other things. I tried it for a while but I ended up just using the web version. All of the stuff that vCenter and more are in the web UI, but the UI is indeed different and some of the stuff is definitely moved around a bit.

14

Okay, that kind of explains a good bit. Thanks for all the help, I'm going to seek out help on ESXi more specifically, but maybe you can help me before someone else in their realm does:

I ended up getting vCenter 6.0 and noticed it seemed to be lacking in some regards. Particularly, I couldn't assign my different ethernet ports to different switches like it looked like My PlayHouse was doing. I had concluded this was because ESXi couldn't assign individual ports on a nic to different tasks... is this true? I remember on an Antergos-based VM setup I was running I could assign any individual ethernet port to a VM if I wanted.

15

You can pass through nics to a vm and dedicate them to a vm in that way, but I'm not quite sure if that's what you mean. I would inquire on the ESXI forum for that.