New guy LLM home lab

Sorry for my answer, but your question is too broad (U$1.000-U$200.000 or more). It depends on what you want to do (inference, finetuning, training, agentics, etc., models size and type) and your budget. There’s a lot of information on the web, set your goals, do your research and evaluate the hardware and budget you need.
You can have lots of performance, but at a price. My two cents
Just as an example to start:
AI Home Server 24GB VRAM $750 Budget Build and LLM Benchmarking
Strix Halo Mini PC vs DIY Build – Does it make sense for Gaming? For AI? What for? - YouTube

High End discussion:
WIP: Blackwell RTX 6000 Pro / Max-Q Quickie Setup Guide on Ubuntu 24.04 LTS / 25.04 - Wikis & How-to Guides - Level1Techs Forums

3 Likes