Talus is an open-source (MIT) inference engine for Strix Halo (Max+ 395/128G).
The engine is a fork of Strata, a C++/HIP engine by Niko1221. We ported Strata’s host and server to Rust, tried a few other popular models, and re-tuned the GPU kernels, which come from Strata and, through it, from ggml/llama.cpp, for gfx1151. It serves one model at a time through an OpenAI-compatible API. A terminal UI switches models and presets, and explains trade offs when changing settings.
Why?
I started chasing the big MoE models that I could fit on this laptop, which spiraled into forking every code base I could find for performance.
The Goods:
HP ZBook Ultra G1a, Ryzen AI Max+ 395, 128 GB, firmware-capped at 70 W, headless Linux 7.0
Qwen3.8-27B, Qwen3.8-Flash-Next, GLM-5.3-Flash, MiMo-V2.6-Flash, Laguna-S-2.1.
Numbers:
(tok/s, end to end through the server, on an HP ZBook Ultra G1a capped at 70 W)
How we got here:
- Kernels tuned for gfx1151: int8 prompt GEMMs, sparse long-context attention, and exact batched verification for speculative decoding.
- Speculative decoding on every model: MTP / DFlash drafters, n-gram and prompt lookup, and “long copy” drafting for edits. The long-copy idea came from gufo; the code is ours.
- Quality comes first:
- A speed change that alters the model’s output ships on by default only if it passes KL-divergence, 60K-81K needle retrieval and agentic-coding gates.
- Everything else is opt-in and clearly labelled. One example: a 4-bit prompt path for 27B passes the gates, but it slows generation after long prompts, so it’s off by default.
What mattered for gfx1151:
- int8 prompt GEMMs;
- unified memory: the 120 GiB GPU pool, and the ‘kho=off’ find that freed about 9 GiB;
- speculative decoding everywhere.
Honest Comparisons (same laptop, measured by us)
- Our patched llama.cpp build:
Talus is ahead on prompt speed for every model we ran on both: 1.3 - 2x on 27B, GLM, MiMo and Laguna. - Decode, chat and code:
Ahead on 27B. On Laguna, chat and code are 2-5% behind. - Gufo:
- gufo is faster at short-prompt prefill on 27B and Flash-Next.
- Talus is faster on decode, and on Flash-Next edits: 108 vs 99 tok/s
- Desktop engines:
Some publish higher prompt numbers on 85-140 W machines: Kyojin on GLM, and lossy 4-bit-activation engines on 27B. On GLM’s prompt speed, Talus is stil about 1.3x behind Kyojin after adjusting for power.
Credits:
Strata - (Niko1221): the base.
ggml / llama.cpp: kernels and formats.
The model teams: Qwen, Z.ai, Xiaomi MiMo, poolside.
Unsloth: the GGUF quants
z-lab: the DFlash drafters.
Gufo: the long-copy drafting idea.
AMD ROCm.
What’s next:
Talus is tuned and tested only on Strix Halo today. The code keeps per-GPU tables amd multi-device hooks, so other AMD GPUs (RDNA4) and multi-box setups may come later. A built-in benchmark and validation tool (quality suites plus SWE-bench with anti-cheating) is coming in the next release.
Github: GitHub - nsbradley88/talus-public: Rust LLM inference engine tuned for AMD Strix Halo (gfx1151): a fork of Strata with ggml/llama.cpp-derived HIP kernels. Five models on one 128 GB machine. · GitHub
HuggingFace: j2ipper