Like the title says.
For around or under 1000 USD can you actually have local AI that is useful?
I know you CAN run local AI on low end hardware, but is it worth it from a performance POV?
Vendor dependency aside, and privacy aside, what is the price-point of ROI with local hardware.
Thanks,
It really depends on your use case. One shot vibe coding useful apps? Probably not. But, there are a lot of tasks like transcription, summarization, that do not require massive hardware investment to obtain, that can be very useful.
It’s all memory limited
~$300 for 64GB of DDR4
~$600 for 64GB of DDR5
32GB VRAM ~$700-$900 on the low side
The PSU, motherboard, CPU, and SSD are just add ons to these anchors of the price.
Ideal LLM setup has 128GB of RAM
so to add some context, i get buy with claude pro today. hitting the usage limit periodically. id be fine with a task taking .5 to 1x longer on my own hardware if it was comparably good output, otherwise id save my money until im ready for a deeper investment in local AI.
If I had to do it under $1k, I’d look at getting a barebones mini PC with Radeon 740M/760M/780M and try to find RAM on sale.
I’m running Qwen 3.6 27B locally and am getting mostly what I want out of it, which is half-decent code local reviews and guidance for learning a new programming language/environment.
It’s not exactly fast (about 8 tokens/second) but its good enough for my use.
I have two graphics cards in my system and am using the llama.cpp Vulkan backend which works just fine with an Intel and Nvidia GPU.
At the end of the day, you need to be able to tell us what you want to be able to do and how fast you need for it to go to be acceptable.
that’s what you use, not what you make with it. Is it coding, writing a book, translation, asking questions like it’s a search engine, all of the above?
Gwen 3.6 2.7 would be totally fine for me. Light hobbyist coding, minor text summaries, etc. nothing crazy. id like to end up with a simple hermes instance that I can start toying with an agent to just tinker with my ai skillset.
I already run audio to text transcribing on a different machine no problem.
something like this? what is a reasonable expectation on tokens per second here?
Maybe if the R9700 Pro AI actually sold at MSRP…
Qwen in the 26-30b class is useful and will run on a 24 GB GPU like a 7900XTX, things like the MoE gemma4 variants are usable on 16 GB GPUs.
I’ve actually been experimenting with it in opencode to write a basic experimental text based D&D/mud style game calling ollama for NPC dialogue in typescript and its been relatively painless and quite competent.
Even smaller models are useful for various things, depends what you want to use them for.
Don’t bother using system memory for LLMs unless you’re on a high end APU, CPU is slow enough on 8b models….
For reference, Qwen3-coder-30b-a3b runs at ~200 tokens/sec on 7900XTX (llamacpp).
You really don’t want to be running that on CPU, it will be painful.
██ ██
██ ██ ▀▀█▄ ███▄███▄ ▀▀█▄ ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██ ██ ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
██ ██
▀▀ ▀▀build : b9957-c4ae9a88f8
model : unsloth/Qwen3-Coder-30B-A3B-Instruct-GGUF
ftype : Q4_K - Medium
modalities : textavailable commands:
/exit or Ctrl+C stop or exit
/regen regenerate the last response
/clear clear the chat history
/read add a text file
/glob add text files using globbing patternwrite me a story
The Cartographer’s DaughterElena had always been told that maps were meant to be read, not lived in. But when her grandmother’s antique compass began glowing with an otherworldly blue light, everything changed.
The compass had belonged to Elena’s great-grandmother, a renowned explorer who had mapped the uncharted territories of the ocean. But this wasn’t just any compass—it was a bridge between worlds, and it was calling to Elena.
As the blue light pulsed in her palm, the air around her shimmered like heat waves. She felt herself being pulled through a dimension she’d never known existed, landing softly on a grassy hill that overlooked a city made entirely of crystallized light.
“Welcome, child of the maps,” said a voice that seemed to come from everywhere at once. A woman materialized before her—her hair silver as starlight, her eyes reflecting the colors of every ocean she’d ever sailed.
“I’m the Keeper of the Compass,” the woman explained. “The ancient maps you see in your grandmother’s collection don’t just show places—they show possibilities. And you, Elena, have inherited the gift of navigating between them.”
The world around Elena began to shift, showing her glimpses of countless paths she could take, countless lives she could live. Some were dangerous, some were beautiful, some were filled with wonder and joy.
But the most important map of all was the one that showed her way home—not to the world she’d left behind, but to the world she’d chosen to create.
She smiled and looked down at the compass, its blue light now steady and warm in her hand. The adventure had only just begun.
Would you like me to continue this story, or would you prefer a different kind of tale?
Prompt: 142.2 t/s | Generation: 202.7 t/s \$\$ \$\$ \$\$ $$
unified memory, depending on model and platform less than 100t/s
VRAM sometimes more than 100t/s
I would put the ideal low end system with 16+ cores, 32GB or more VRAM and at least 64GB of RAM ~$2500
Can you run LLM’s for less, hell yeah, a good used setup with an accelerator or an MiniPC with lots of ram, you could get away with ~$1500
For the upfront cost you need to know what you want to accomplish with the moeny spent. If you have goals, find out what they are worth and decide on the pragmatism that you will get your investments worth.
Personally I had a GPU worthy before it kicked off, so I haven’t felt the pain of modern prices, but I’m aware of my options, and the R9700(~$1300) is truly the sweet spot.
Of course someone benchmarked it already 
Strix Halo is newer than the RDNA3 APUs I was referring to and it looks like fast memory is being used here but 20-40 t/s sounds about right for other APU setups as well.
Ultimately, discrete GPU will be faster but limited on VRAM, APU system will be able to run bigger/better models but at a lower average token rate.
I personally prefer the faster token generation of dGPU, but having to buy a second one to fit decent models also put the budget well over $1k.
Maybe @lhl can offer some more advice about the APU setup
128GB Strix Halo systems are $3-4K atm (and were never much less than $2K to being with, so regardless of how useful they are (current llama.cpp benchmarks) they’ve never really been a <$1K option. With a Strix Halo (or DGX Spark) can run decent quants of ~120B models (not much really worth running atm IMO) or low bit useful quants of ~200B models (DS4 Flash), but IMO Strix Halo is best for having a multiple models loaded and using a 30B class model as the workhorse/orchestrator.
For coding or agentic work, right now I think the sweet spot is the ~30B size. You can run a Q4 quant of Qwen 3.6 35B-A3B or 27B (dense) w/ full 256Ki FP8 KV on a dedicated 24GB GPU. You should be able to get used RTX 3090 (better option) or 7900 XTX for ~$800 or less. That and going to your local computer recycling center or checking local postings and seeing if you can pick up cheapie PC with PCIe 4.0 and a decent 700W+ power supply, would be my <$1000 recommendation.
When it comes to useful local AI, there’s actually quite a bit of stuff that doesn’t need huge amounts of VRAM - there are excellent small translation models, there’s a lot of interesting voice (ASR, TTS) - having real-time translation and transcription is really underrated in terms of utility. There’s a lot of interesting OCR models. There are small computer use, categorizer models as well. Smaller models are constantly getting smaller/more capable (and importantly), more reliable at specific tasks.
What would you say is the best OCR model right now? I’m very happy with Phi-4 but accuracy suffers at lower precision, I think 32GB VRAM is probably the lowest I would go for this one.
I use deepseek-ai/DeepSeek-OCR · Hugging Face on my 16GB GPU, I don’t process a LOT of files, but it works when I use it
Here’s the olmOCR leaderboard which might be a nice place to start since you can restrict by size:
OmniDocBench leaderboard:
Also, an aggregate of a bunch of vision benchmarks:
and
UPDATE: btw, was noticing a lot of disagreement on “best” ocr model. Ran a review: ChatGPT - OCR Benchmarks Analysis - its conclusion is the Apache licensed infly/Infinity-Parser2-Flash · Hugging Face (2B) and infly/Infinity-Parser2-Pro · Hugging Face (35B-A3B) is currently the best.
Lots of different answers to your question…
If you do not care about performance, you can run LLMs partly or fully on your CPU. You will not get great numbers, but it will run.
You could live within the limits of your video card. Might be good enough for your purpose. Might be a non-starter. Depends on the card you have, and what you want.
You could upgrade your video card to one well-suited for running AI. This could blow through your $1000 budget pretty easily.
You could add another card as your GPU for AI. There are lots of interesting alternatives on the used market. But for most desktop CPUs/motherboards, you are going to run out of PCIe lanes.
I started with a used HP Z820 that I bought several years back. Dual Xeons mean more PCIe lanes, so enough to easily host an added card. As the box is wonderfully obsolete, upgrading CPU ($70 for 16-cores) and memory ($90 added 64GB to total 256GB) was cheap. Yes, the old gear is slower, but in present the performance per dollar is better. You do not need the added CPU or memory, so much.
Bought a used AMD Instinct MI25 (datacenter GPU). This card has 16GB of GPU memory, and seems to churn through LLMs at a respectable rate. Present prices on the used market are <$100, but takes a bit to get going.
Named the box “beast”, and connect over the local subnet from my desktop.
You might be able to fit one of these cards into your desktop.
My misadventures:
I have great OCR results using Qwen3.6 35B A3B UD-Q6_K_XL on my W7900. However the 9B model is also really good, they all have the same mmproj vision encoder so it’s just about how the model organizes it.
As far as the OPs question, I personally use a W7900 and a 7900 XTX as my two daily driver cards. I just got an R9700 as well, but I haven’t set it up. Other than that, I have a system that’s 6800 and another with a RX9070. Overall, the performance on all of them is quite good, depending on what you’re doing. If you really want to use any kind of programming in a logical sense, you need to use a 30B class model. Right now, the absolute king of this is the QWEN 3.6-27B model, especially the dynamic quantizations from Unsloth. You can combine that with Unsloth Studio and you can post train it some if you need.
For a MOE model, like the 35B A3B, QWEN 3.6, I get up to 110 TPS generation and 1250 TPS pre-fill on the W7900. It’s actually a slightly faster on the 7900XTX, but the quality is much worse because of the quantization I have to run for that model. However, the 27B is quite good with the 4-bit quantization, especially again the dynamic Unsloth version. And with that, I have around 64K context with Q8_0 KV. 40ish TPS generation and upto 600 TPS prefill.
On the RX 9070, I typically have used that for search-related duties with a GEMMA-4 E4B model or another smaller model, and I typically get over 1500 TPS pre-fill and 130 TPS generation. Without MTP.
If you already have a computer that can accept a good graphics card, I would invest in the R9700. If you can also invest in a RX 9060 (to use the same drivers) or even an Intel arc A310 for video output, this would allow you to use 100% of the VRAM for AI Compute.
However, it depends exactly how much development you plan on doing and how often you will use this and what the token performance is that you are looking for. If you are truly hitting your usage limits on a Claude Pro plan, then investing 13 or 1400 dollars in an R9700, I think it’s worth it, versus the sub 1000 dollar options, unless you can fit all your work into 24 gigs and find yourself a nice 7900 XTX.
I didn’t expect this thread to get the engagement it has and I thank you all very much! my conclusion is that more or less Local AI under 1000 is possible, but not a great fit for my use cases. Probably keep an eye out for the occasional surplus deal or something used but not really interested in anything “new” in this price range.