Rookie here looking to build a decent and affordable ($2000) AI rig.
After lots of back and forth with google AI, the title says it all.
Hoping to leverage both platforms for a decent hybrid setup —
Nvidia for generative AI, fast tokens, average tasks.
AMD’s 128GB unified memory for exploring 70B models for accuracy and reasoning.
What do you all think, can Vulkan bridge the gap?
Oops, just realized I typed Max 400 instead of 300 (Gorgon vs Strix Halo).
While 192gb might be awesome getting larger models with less quantization, I’d think the limited 256 gb/s memory bandwidth would slot token generation to an unbearable crawl, wouldn’t it?
Well I just read through Wendell’s Franken Strix Halo post, and it answered a lot of the implied questions I have in my post here, thank you sir! Lots to learn in this new venture of building a local AI rig for sure. I may need to splurge a bit on the Nvidia external GPU I get, and it’s too bad I won’t be able to use more than one eGPU due to PCIe constraints on the Strix motherboard options, as far as I understand at this point.
I particularly liked this part:
A 120B MoE at 100 tok/s on two 32 GB cards is a genuinely usable local-inference box for coding, agent loops, or anything token-hungry. That’s the standout result. More to come there.
The iGPU is a capacity multiplier, not a speedup. Use it to fit models that don’t fit in dGPU VRAM (the 176.9B Flash-Next runs at ~26 tok/s across all three devices, and at ~24 tok/s on the iGPU alone with the fork). Don’t expect it to make small models faster.
If time-to-first-token on long prompts matters, spread the model across GPUs. Prefill scales with device count even when decode doesn’t.
The 3-device split unlocks the model’s full native context. With 8-bit KV it runs Qwen3.8-Flash-Next at its full 262k window with 100% needle retrieval long-document and whole-book-in-context workloads are a single prompt, no RAG pipeline needed (see Section 2).
External PCIe switches are a real bottleneck for GPU↔GPU traffic. If you need tight multi-GPU collectives, use a platform with native GPU-to-GPU paths. For llama.cpp-style layer-split inference the switch is fine.
On HIP, the iGPU can’t read dGPU memory (the runtime advertises peer access that faults in practice). Device order matters: iGPU first in the pipeline, or prefill crashes.