Homelab AI Expansion Thoughts?

Hardware Hub
, ,
1

Contemplating expanding my homelab AI solution, and would like to get some input from others on my planning thoughts. Currently running a dedicated box with below.

Ubuntu 26.04 LTS
5070 16GB VRAM
64GB RAM
10G LAN
Use cases are Ollama, Open-WebUI, Hermes, code assistance, and a bunch of automation tools like n8n that use ollama externally. Can’t do video generation or some of the better and larger models which is a big blocker right now.

The issue is pretty clear, not enough VRAM. At this point, I have 4 solutions with different costs:

  1. Buy a used Bitcoin mining frame, get a nice server mobo that has 4-8 PCIE x16 slots and scale out GPU’s as wanted and based on price
  2. 2 x NVIDIA DGX Sparks with direct connection in a cluster
  3. 2 x AMD Halo with direct connection in a cluster
  4. 2 x Minisforum boxes

Would love to understand other peoples experience and appreciate any lessons learned. Thanks.

2
  1. Probably not easy but certainly a learning journey. You’ll need risers, and those PCIe switches. Only do it if you got deep pockets or hardware is reasonably priced locally.
  2. Slow for dense models, works fine for MoE models. Start with 2x for sure, run DeepSeek v4 Flash 0731.
  3. Clustering over USB? Eh, get a DGX Spark
  4. Not sure which ones
3

The answer depends on your budget and goals. Obvious answer is before committing to building something, just rent the hardware hourly as needed to compare. You can rent sparks for example, see what it does well or sucks at. Get a feel before sinking thousands of dollars into a hardware project.

4

I’ve been deep diving on my use cases across all 4 options and from my reading, seems 1 and 2 are the most supported options. And neither are cheap for sure but 1 allows for adding resources where 2 is fixed unless I add another $5K/device and the concern on scalability there. Biggest hurdle atm is getting to 60-90GB of VRAM which is solvable with 1 of those 96GB VRAM cards but, $15K? No thanks. Still digging through supportability but welcome counters to my ideas here. Thought short term I could get a few used 3090’s and a used mining GPU frame on ebay and at least have a solid upgrade from where I am right now. Seem logical or am I too deep in my own assumptions?