My apologies as this will probably be a very long post…
I currently run my home lab on a number of old enterprise servers (three DL380 G5s and a PE2950 G3 along with 1 TrueNAS box and 3 QNAP NAS boxes) currently setup in 2 42U racks and a 45U rack. Which gives me lots of space for new gear and or modifications it does take up a good chunk of room in the garage. The only other equipment in any of those racks is a monitor, a mid-tower configured as a testing machine, KVM, modem, MoCA adapter, 3 managed switches, Unifi AP and some UPS boxes. In the fall of 2020, I was looking at upgrading my compute machines and my storage machines to newer and more capable machines, and eventually changing out all the switches to Unifi 10G switches.
However, this Christmas I got three Raspberry Pis to play with and I started wondering if I could migrate most of my stack to them. I moved from Linux VMs running native services (things like Nginx, Apache, Emby, and other web-based services) to running things in Docker about a year ago and loving it ever since.
I have checked and most of my software stack will run on ARM64 but not everything so I will probably keep a single X64 system for running things like pfSense, Mailcow, ProxmoxVE, ProxmoxMG and others that require X64.
Could I not get say a number of Pis together and use something like a docker swarm or Kubernetes to build something that could run my stack. Ideally, The Pis would be rack-mounted and require little configuration to add them into the cluster and could be removed with little work as well. I could either go with an SSD on each one for local storage for the docker containers and the network storage for the rest or should I do something that only uses an SD card for the Pi and network storage for everything else.
I would ideally like to trim down the size of the racks to something that would be about standing desk height and also cut the volume of the machines as the enterprise gear is quite loud. Though I am not sure about moving away from my plans to swap the QNAP boxes out for TrueNAS boxes in either enterprise machines or custom-built machines (looking for 12 to 16 bays for storage and 2 bays for OS).
I am looking for some guidance on which way to focus my learning to build a test deployment using my 3 pis as a proof of concept. As well as any other advice, guidance, input, comments that will help with these ideas I am working through during this planning phase.
PS: I hope I picked the right topic for this post.