Servers for a home lab / home network

Hardware Hub
1

Looking for a few servers for a home lab / home network setup.

General things i’d like to learn / setup are:

  1. a nas box
  2. steam cache
  3. replace my current router and wireless setup with pfsense or other router os and a couple of wifi ap.
  4. a machine or 2 for learning vm on.
  5. any other stuff worth learning.

general requirements for hardware is cheapish. not looking to break the bank but don’t want anything that is just worth what i can pay for scrap at the local recycling station.

Any suggestions for where to look / get a good deal on a few machines?

2

Pawn shops. Take a USB stick with a distro to test with.

3

Get a (or more) old-ish (approx 5 yrs or so) workstation, like Dell, HP or IBM, fill it with as much RAM as your budget allows, add an SSD or NVMe (if applicable, but for that age probably not yet) for the OS and at least 2 HDDs for storage in RAID1.

Start with 1 machine, determine which role suits it best from your list and finish that one before adding another project.

HTH

1 Like
4

Plenty of cheap* server hardware out there (cheap is relative term)

If you live in a megatropolis like LA or NYC you should have actual IT-specific recycling places you can go to lay hands on the gear before buying it.

Check nearby universities and government types for surplus sales.

If all else fails, ebay or Craig’s list.

Go with Dell or Lenovo (IBM)… HP / HPE is nice gear but firmware updates and some drivers are paywalled.

2 Likes
6

Thanks folks for the suggestions. I should have clarified my location in my original post as being in what is considered country side for my area. I live in north eastern part of Connecticut and as such I am an hour drive or more from most metropolitan areas. HTH.

7

MA to the North and NY to the South, man you’re in a prime spot. Maybe folks around the Valley would have better local sources. Tons of huge well funded schools around there, lots of them heavy into the computer sciences. I get there’s a lot of traffic, so maybe take the train and then do a one-way rent a truck if you buy something big.

You’ll find much better deals at local surplus sales / auctions. Buying stuff “online” is easier I guess, but you’re going to pay a lot more.

8

I can’t tell you where to look, but I can give somethings to take note for each of your points.

Get a Synology 2Bay+ in RAID1, SHR, or SHR-2, or some other brand NAS. Generally you don’t want to really play around with this too much if you’re looking at high uptime, high reliability data storage. Synology ain’t cheap is the biggest problem you’ll run into.

If you have a reliable NAS, you can SMB/NFS mount it either a Hypervisor or straight to the VM and offload the storage that way. Something like LANcache should fill your needs.

There is no cheap way to do this if you want routers + aps separated. The brand of mini-pc I use at home is Qotom if you’re looking for something, as for APs, check if your old router has bridge-mode or do some research into getting some consumer APs with bridge-mode capability.

VMWare vSphere Hypervisor (ESXi) licenses are free and proxmox is also a good option, if you properly plan out how you want to do your network, you can just use working scrap machines to get yourself a virtualized environment. Mine is literally stacks of old laptops running ESXi.

Raspberry pis.

With all of that, buying cheap “for parts” stuff off ebay can sometime be fruitful if you’re willing to put the time out to drop things off at a electronics recycling every once in a while.

9

Pretty sure Lancache has issues with at least SMB, and probably NFS as well. iSCSI would be fine though.

Older intel parts, even back to socket 1366 can still be pretty powerful, and can fit plenty of ram, well, and also ECC DDR3 is cheap. The downside is that they suck quite a bit more power than new chips do for the same performance level.

The balance of which generation of older chips to get is between paying less now, and paying more later in power, or paying more now and less later. It partially depends on your power rate.

It is more applicable the more machines you are running. One box on 24/7 is not nearly as much of a difference as a whole rackful.

10

Hyper-V is also free… Microsoft trusts you to not (scouts honor) install the gui version of the server if you don’t have a license.

It’s certainly not the most popular environment for in-house shops but it is gaining traction in the cloud somehow.

11

I don’t think it is possible in server 2016 or newer. Even with the full-fat license version, if you install in no GUI mode, to then switch to GUI mode, you have to reinstall.

Server 2012 (r2) might be different, IDK.

12

hyper-v core still free far as I can tell… the gui management tools run fine under w10, or, great time to learn powershell.

13

Hi, I am currently in the process of planning a home server as well. Here are some of my ideas and thoughts, hopefully they are helpful to you and others.

Disclaimer: I am a beginner myself. Please let me know if something is incorrect, might otherwise be problematic or if there is simply a better way to do something.

How should it be done?

I thought about different ways to do it. In the end, I came to the conclusion that, especially considering that electricity costs upwards of € 0.30 ($ 0.35) in Germany, I will try to consolidate as much as possible in one machine and use Proxmox as a host OS. At least in my situation, I think that it is better to spend a bit more on one well-specked machine that can run multiple VMs, instead of spending money on multiple machines that have old and power hungry hardware.

However, in my opinion, Proxmox is the way to go anyway. I think it is a “skill” worth learning, you can easily set up VMs for testing purposes, and it is relatively easy and convenient to manage your VMs via the web user interface.

In Proxmox, I think I will probably create a single VM for every major service I want to run. This should make everything more resilient, since a misconfiguration or some other issue on one VM does not affect the others (of course, running everything on only one machine is also a risk). If you use distros that are light on resources (especially without a desktop environment if possible), you also should not run into performance issues without a Threadripper CPU.

If you do not want to use Proxmox and instead use multiple physical machines, I suggest getting some Raspberry Pi 4’s (the 2 GB RAM base model should be fine for most use cases). I found this YouTube video to be a good tutorial on how to use a Pi as a basic NAS.

I also considered Docker, but each port can only be assigned to one Docker container, which is troublesome if multiple Docker containers use, for example, port 80 for http. This website describes the problem and how to solve it, but I still decided that I am going to use Proxmox for my server, also because Proxmox has support for Linux Containers (LXC).

What is the server supposed to do/be?

1. Router

I am currently using a Rasberry Pi 4 4 GB as a router, with a USB dongle as second NIC. Alternatively, you could also create a pfSense VM on Proxmox (I think this tutorial might be helpful).
If you want, I can also provide instructions on how I set up my Pi as a router (it is surprisingly straightforward if you just want NAT and DHCP).

2. NAS

I am planning to use an OpenMediaVault (OMV) VM on Proxmox. It can not only do RAID, but also supports ZFS and it can create volumes that are recognized by macOS as Time Machine backup volumes. OMV also comes with support for Docker containers (this way, you could run some or all of the services you want in Docker containers).

3. Private Cloud

I think I will set up Nextcloud on a VM in Proxmox. If you do not have a static IP address, you will need to configure DynDNS to access your cloud from outside of your network.

4. Other stuff

Besides this, I have some long-term goals (all as VMs on Proxmox):

  1. MotionEyeOS
  2. PLEX
  3. Home Assistant with Grafana plugin (for lights and sensors for temperature, humidity and other air quality metrics like CO2)
  4. Windows 10 VM with GPU passthrough for gaming

–42069

14

thank you for the input JackHerer. I’m not opposed to 1 or 2 well specked machines and do things thru vm’s. vm’s are something i’d like to learn as it’s the way thing are going. I’d like to keep my router separate from my other machines as i don’t want to disrupt the other people using the internet while mess with another vm on the same box. I had not considered a raspberry pi as a router and will have to look into it. Thanks for the suggestions.

15

This is not quite correct. If you make a virtual docker network, then multiple containers can have the same port, but this network is only accessible from the docker containers. And you can always remap the ports, so if the docker container wants to use 80, you can let it have 80 inside the container, but on the host give it 8080 (or just about anything else), then instead of accessing it like http://127.0.0.1/ you acess it like http://127.0.0.1:8080/, which is not an issue for home use, although it would not work well if you wanted to run a public website.

Very good idea. If you screw up the VM that the router is in, well then you wouldn’t have internet access, and then it might be interesting getting the vm rebuilt.

I’m not sure it would have enough throughput, although I’m not sure what the PI 4 can do.

16

Thank you for that clarification, but is that not the solution which I linked to in that paragraph? I guess next time I should not just provide the link but also summarize its content in my post :slightly_smiling_face:

As I am using my RPi 4 as a Router, I can definitely say that it is powerful enough to max out a gigabit network without overclocking. It also almost maxes out the CPU at full gigabit speed, too, but a small-ish passive heatsink seems to be enough for me (and something like the FLIRC or “Armor Case” should definitely be enough even for continuous max load).

In that case, here are my notes on configuring a RPi with Raspberry Pi OS as a basic router with NAT and DHCP, based on this original tutorial, which I altered to suit my needs. You will need a second USB NIC (eth1). eth0 will be the WAN port and eth1 will be your LAN port.

Step 1

If necessary, comment out eth0 in /etc/network/interfaces so that it is managed by network-manager.

Step 2

Install required software:

sudo apt install dnsmasq
sudo apt install dhcpcd
sudo DEBIAN_FRONTEND=noninteractive apt install -y netfilter-persistent iptables-persistent

Step 3

Configure eth1 in /etc/dhcpd.conf:

interface eth1
	static ip_address=10.10.30.1/24

Step 4

Backup original dnsmasq.conf:

sudo mv /etc/dnsmasq.conf /etc/dnsmasq.conf.orig

Create new dnsmasq.conf and insert the following text:

interface=eth1
no-dhcp-interface=eth0
dhcp-range=10.10.30.2,10.10.30.100,255.255.255.0,24h
dhcp-option=option:dns-server,1.1.1.1

Step 5

Uncomment the line net.ipv4.ip_forward=1 in /etc/sysctl.conf to enable port forwarding.

Step 6

Add NAT rules for eth0 and make them persistent:

sudo iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
sudo netfilter-persistent save

Step 7

Start dnsmasq:

systemctl enable dnsmasq

Step 8

reboot

17

Yes, the article is correct, but the statement "but each port can only be assigned to one Docker container" is incorrect. It would only be correct if you added something about this only applies to port mapping to the host.

Not a bad idea. Helps people not have to click a link if they already know about what the article talks about.

Yes, but are running a OpenVPN server or doing QOS or running Snort/Suricata on your router? All are fairly popular to run on router (at least for people that would do DIY router) and add some overhead, which probably not there if you want to do 1gig on a PI 4. I’m not saying it doesn’t work, or wouldn’t be enough speed for most people, I’m just saying that it should not get a flat recommendation, people should do more research first.

18

Got it, that makes sense.

True, and if the link does not work anymore, at least some information will still be available in the post.

Thanks for pointing that out, I was just assuming a similar use case to mine (basic router which only provides NAT and DHCP without any bells and whistles). As for OpenVPN, QOS or other things, I have not tested it yet on the Pi and I am not sure about the impact on performance. Practically speaking, it should not be a huge issue, since most VPN servers, let alone internet connections, will not offer gigabit speeds anyway. Of course you are right in saying that there are still plenty of cases where the RPi’s SoC could be a bottleneck (even after overclocking) and may not be adequate for every use case. If I come around to it, I will do some performance testing and post the results in a new topic which I will link to here.

1 Like
19

Exactly.

That sort of thing is always welcome around here.

20

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.