Can I get some advice and some ideas for homelab?

Hardware Hub
, , , ,
1

Hello!!!

This may be a bit of a long post, but please bear with me. I am new to the homelab world and want to expand my horizon in the field of technology. I have some hardware I am looking to set up in a homelab, but I am not entirely sure what should go where, and if there is anything I could add to it that would add functionality to my life in some way.

I have tried to look into this online, and I have watched some videos on YT, but most of them are “…and install Proxmox” or Kubernetes, or some other top-level software that doesn’t really go into explaining what they are using it for, why they are using it, or how it benefits them, and that is what I need, the what, the why, and the how.

Below, I will mention what I want to do over time, and then after that, what hardware I have lying around, and just try to get some general input.

I am still a newbie in this space, so please keep that in mind as I ask questions.

Want to do:

Networking-
On a single box, I want to have pfSense, Pi-Hole, Tailscale, nginx, and some sort of network monitoring too. The idea is to have better management of the network, have access to Plex throughout the house if there is an internet outage, and access some of my stuff from off-network if need be. Would like some good network monitoring tools that work well, are easy-ish to set up and maintain, and people here have experience with if I have questions. This will run on a Lenovo M720Q.

While I am here, I assume that once I have pfSense set up and have a WAN and a LAN network, I would use LAN for things I want access to only on the network inside of my house, so things like cameras, but things I need access to outside of the network would be on a WAN? If that is the case, can I set it up to have things like Plex or other things like a camera network to work on the local network if the wireless network goes down in the event of an internet outage?

I want to learn clustering and high availability, but I want to do it in a low-stakes type of way, so I am thinking about running a couple of Minecraft servers for my kids in a Proxmox or Kubernetes cluster. I am not sure which is the better way to go, but I do want to learn both cluster types.

Other than that, I want to be able to do the following.

  • Immich Server to replace Google Photos
  • Cloud server for document storage and retrieval. Mostly for kids’ school stuff or documents I want to hold onto and have access from all of my devices. I want to remove the need for Google Drive
  • Password Manager
  • Backup server that I leave at work to back up media and documents. Would be a large storage pool to cover Plex and other media, as well as immich and cloud server backups.
  • Home security to replace Ring and Google cameras.
  • Audiobooks service like Plex to replace Audible.
  • Music service like Plex to remove Apple Music.
  • Home assistance to monitor everything in the house, and the local host thinks like a thermostat.
  • Ark Survival Ascended servers (ideally 8 in total, but RAM prices, am I right?)
  • Kali environment to mess around with and learn the OS.
  • RSS feed manager that can push content and updates to different channels in Discord servers.
  • AI that can handle an LLM, something like ComfyUI for video and photo generation, maybe a music AI, and something that can do deep research for me in various topics. I also want to train an AI for specific use cases to learn that process too.
    *Folding@Home server to maybe help contribute to science and stuff.

Anything else you think I could add?

Now, for the hardware I have lying around that I am not currently using.

  • 2x 9950x3d, each with an X870E motherboard. I am thinking I could use for 2 AI servers or an AI server and an ASA server to run 3-4 instances of ASA. I have no extra RAM atm though, so these could sit here for a few months before I use them since I would want at least one more 5090 for an AI machine. RTX PRO 6000 is the dream, though.
  • 5800X with motherboard and 32GB of RAM
  • 6700k with 32GB of RAM and board. Just upgraded my Plex server to a 5800XT and want to repurpose the old hardware.
  • 3x M910Q with 16GB of RAM. I am thinking about using this to mess around with clustering and HA with a couple of Minecraft servers.
  • M720Q with 32GB RAM, planning on using it as a networking server.
  • M710E with 8GB RAM, and it needs a new PSU.

I do not really want to sell anything, but I do want to try to use everything that I have plenty of time to build up my homelab.

Thank you all in advance, and if you need any clarification, please let me know. If I feel like I am pushing back on any suggestions, it is only so that I can get clarity and have answers to questions. Again, I am new to homelabbing!

2

That’s the important bit.

Reading your post it sounds like you have plenty of ideas of what to do… eventually…

My recommendation is to start out with a single pc running 24/7 with an operating system you’re familiar with.

Next every time you do something on your regular computer, you start by asking the question “Could my server do that for me?”. Ordering a pizza is probably easier just to do on your regular pc. Backing up family photos to a cloud service, that sounds like something the server could do at 3am when no one is using your internet connection anyway. Or how about downloading the newest microsoft patches so that you and your family can upgrade from your local server rather than all of you needing to download the patches from microsoft?

At some point you’ll stop wondering what you need a server for and start wondering how you ever got along without one.

Then you install that minecraft server your kids need and soon after you realise that the server is no longer yours as you get hell from the kids when you try to update and reboot the server - so you set up your second server for you to mess around with while the kids play minecraft on the original one.

You add a bunch of drives to the second one and add your entire movie collection. Before you know it, you’re catching flak from the wife if you mess with server two while shes watching her favorit series on the tv…

Third server, here we come… Well, actually you add server 3, 4 and 5 and cluster them and start running stuff as vm’s so when you need to take server 3 offline for maintenance you migrate the vm’s from server 3 to server 4 or 5 so no one will notice the downtime.

Start out small and expand from there, let it evolve naturally, sounds like you’ve got plenty of equipment. You won’t get everything right in the first try anyway, I started out with a single windows 2000 server long time ago, evolved past a bunch of old enterprise servers in a rack with a fibre channel SAN, then got hit with the electrical bill from hell and now I’m experimenting with a ceph cluster build on thin clients and I’m not done yet.

The real goal is to end up with a bunch of virtual machines that has a way higher uptime than any of your physical machines… :slight_smile:

2 Likes
3

First thing you need to figure out is this; Which services do you need to have on 24/7, and which services are more “Eh… Guess I’ll train that data for the next two weeks and see how it goes“ and then just let that hardware idle for a couple of months, before the next project comes along.

Ideally, you would like to draw as little power as possible on your heavy computational servers. This isn’t 2010 anymore, where cores come at a premium and hardware can barely run two demanding programs at the same time. These days, a 12 core Ryzen 7900 is a popular homelab option for many, due to it’s 24 threads and low power footprint at only 88W when worked to the bone.

That single CPU can run 99.99% of all your 24/7 stuff and then have plenty of room left over for more demanding tasks.

Furthermore, a threadripper or high end Ryzen workstation, can provide most compute that you need, especially now that virtualization and WSL is a thing. This allows you to do a lot of virtual workloads, which wasn’t available earlier.

If what you need is not so much workhorses, but network endnodes, a few Raspberry Pis (or Odroids or whatever you fancy) or Intel NUC machines (basically small desktop boxes with a laptop CPU) can make for very good simulators.

From my own personal experience, a file server, personal HTTP server and a film/streaming server are the only things I need 24/7 - and those could easily run on a budget 4 core computer. I have set up a compilation server as well in the file server though, since a Ryzen 7900 is such a beast for what you get from it. Nothing beats playing a quick sequence of FFVII: Rebirth while I am waiting for a compilation to finish… ^^

Everyones use case is different though.

1 Like
4

I’d not do this. Mixing work with home has blown up for some people in awkward ways.

5

What are some of the cases that might occur? I own a repair shop and have the space, so I figured that would be a great place to store it, but maybe I am overlooking something?

6

I think ack meant more for someone in an office environment that isn’t their own business. You would be able to control physical access to the machine and it would be on your business network which may not be the most safe for the business info and personal info to be available over the same network? I am just speculating on the last part. Maybe setting up a VLAN at work to separate it from everything else could be an option. Working towards something similar with old hardware I have sitting around.

1 Like
7

Its not clear to me what your current setup is, but I would start with storage and virtualization first.

For storage you would ideally use separate physical machines. I use truenas but there are others to try. One for your home and one for a second location if you can. Make sure the second location is encrypted and locked down. Truenas loves RAM (8GB minimum for a backup target - at least 16Gb for a primary instance), but processor wise its pretty low burden to handle a few HDDs. pick something that is old, low power, or both to use for this.

For virtualization I would try proxmox and XCP-ng. I use proxmox but its got its own problems and if you are thinking of doing HA or clustering XCP-ng may be better from what I have seen. you can then install LXC’s and VMs in there. I would put most containers inside a single VM. separate when required and run LXCs when you need a standalone service.

I haven’t done anything with AI self hosting yet, but you have plenty of hardware to try it separately. once you have most of your services up and running you can address hardware changes as needed.