I began a home lab journey about 4 years ago, bought various bits and pieces over the years, but it started to come together only recently.
The company I work at decided that in 2025 it would officially start investing in AI. A couple of us got IBM Watson training, a handful of Copilot Enterprise subscriptions for evaluation, and by end of year we got a green light for the 2026 budget to include AI hardware aka high memory server video card or cards and more subscriptions.
The company has strict “no cloud” policies. Only Teams runs externally, after Skype for Business got killed. With AI it became increasingly worrisome what gets into the ChatGPT prompts by devs. We have yearly ISO audits and new laws require a detailed documentation of all IT systems.
Upper management was tasked with researching how we can incorporate AI into our work. Some went with evaluating coding services, I went with local hardware and LLMs. We have an internal project for gathering metadata on our thousands of contracts.
We process the documents with Python, scrape for the contracting party, the date and nature of the contract, categorize them, etc. Our current estimation is 2 weeks of crunching with an adequate card for all the documents to be processed. We run GPT-OSS:20b because that fits in vmem.
My colleague said Nvidia is the only way to go, so I went on eBay to look for RTX Pro cards I could afford, and bought an RTX A4500, RTX A5000, and RTX 4000 Ada. Based on a L1T video I also got a brand new Intel B50 and an RTX Pro 2000 Blackwell, just so I could say I have a Blackwell card.
My primary goal was (and still is) to find suitable cards to recommend to the company for purchase. My secondary goal is to utilize the hardware for use in personal home projects like as coding agents and analyzing video footage from the cameras on our property. And of course to learn as much as I can on the subject.
Wife runs a bakery/caffe in a separate building on our property, I want to run patron statistics using AI. Started a Python PoC to analyze the security cam footage for how many people come, what time of day. I want to have statistics to help the business. See daily/monthly peeks and trends, to better plan logistics. It would be a lot cheaper if I didn’t need to power many cards.
Proxmox was my platform of choice, because I was already familiar with it on a single machine. I wanted to extend this knowledge to multiple nodes. I already had a medium-sized Threadripper server, and a bigger one ready to go. During the Christmas sale, I got a barebone 12600H Minisforum MS-01. What I want to learn is how to migrate VMs while still providing some basic AI capability.
The Flock was born: Albatross, Condor, and Kestrel. All of them run Proxmox 9.1 and OpenWebUI in LXC for each card.
Albatross:
Case: Fractal Design Define 7
PSU: Seasonic Prime TX 1600W titanium
Motherboard: ASRock WRX80 Creator
CPU: Threadripper 5955WX 16 Cores
Memory: 8x 32GB 3200 Kingston DDR4 unbuffered ECC memory (KSM32ED8/32HC)
Storage: 240GB Kingston DC1000B, 15.36TB Kioxia CM5, 2x 480GB Seagate Nytro 1551, 2x 4TB WD Red HDD
Video Cards: Radeon Pro WX3100 4GB (for install), RTX A4500 20GB, RTX Pro 2000 Blackwell 16GB, Intel Arc Pro B50 16GB
I will also be running Windows and Windows Server VMs on it for testing and providing my friends/colleagues with local environments.
Condor:
(I swapped the cards already and made a gap between them, don’t worry)
Case: Fractal Design Node 804
PSU: Be quiet! Dark Power 14 1200W
Motherboard: ASRock X399M Taichi mATX
CPU: Threadripper 2990WX 32 Cores
Memory: 4x 16GB 2666 @ 3200 Kingston DDR4 unbuffered ECC memory (KSM26ED8/16HD)
Storage: 2x 480GB Kingston DC1000B, 7.68TB Kingston DC1500M, 2x 8TB Seagate Exos
Video Cards: RTX 4000 Ada 20GB, RTX A5000 24GB
I was considering swapping out the 64GB memory with 128GB which is the board’s max, but it would be an unreasonable expense right now, plus it matches with Kestrel’s 64GB so it’s a good challenge of constraint.
Kestrel:
Minisforum MS-01
CPU: Intel 12600H 4P/8E Cores
Memory: 2x 32GB 4800 Hynix DDR5 SODIMM
Storage: 480GB Kingston DC1000B, 4TB Crucial P3
Video Card: RTX 2000E Ada 16GB
The memory I got from eBay a day before Christmas for €340, which was a pretty good deal considering… and is still to arrive. The RTX 2000E is also arriving next Monday. I specifically got the MS-01 because it has 10 gig ports which I want to use to test network speeds in the family houses. Contractors used Cat6 instead of Cat6A and I wanna know how bad it is.
I have another 15TB CM5 which I plan to swap out for the 7TB in Condor, and put it in Kestrel if it will fit.
I also thought about getting some 22110 enterprise SSDs for each system from eBay, buuuut stuff is already running and good enough with what I had on hand. No need to spend more money.
In order to stuff bigger and better models into vmem, I had my eye on a 32 or 48 GB RTX cards, like the RTX Pro 4500 or 5000, but that probably won’t happen for a while as I need to stop spending money. I forked out a lot for the new 5955WX, the new PSUs, and I also bought a bazillion Ubiquiti gear (switches, firewalls, access points, cameras).
If I managed to score the 32GB Radeon R9700 that would be a great opportunity to try a Radeon card in AI workloads. Looked at some videos and apparently it’s good with image generation. My wife would love a high performance image generator for her sticker and t-shirt print side-business. Eventually I would also like to sell off the cards that “didn’t make the cut”, and the highest memory models would stay as the video crunchers.
I also got a Beelink GTR9 Pro 128GB for work, just so I could stuff GPT-OSS:120b into memory.
Some of the stuff that got phased out:
Threadripper 1920X, 4x 480GB Seagate Nytro (wanna get a compact SATA backplane for the Node 804) , 2x Seasonic Focus PX 550W, Beelink SER9 32GB which was my previous work PC
In the back there is my NAS running TrueNAS Scale in a Jonsbo N4, Intel 12100T, 4x 12TB Seagate Ironwolf Pro, Kingston DC1500M 3.84TB, Intel Optane 905p 480GB