Curious about the above given the below news:
XeStrata:
As the only remotely reasonably priced 32 GB card is an Intel Arc, I’m curious about this from a hardware savings perspective.
On my system (E-2478, 64GB RAM, B580, configured with qwen3.8-flash-next-iq2_xs) after a few chats it worked at around 25t/s. Just a few minor tweaks (so it would build with docker), waiting a few minutes, and it just works.
I really liked the setup process, I don’t know much about all the different options to run one model, so having them explained and even recommended based on my system was a nice thing!
Now I have to throw some stuff at it and see if the output is usable
1 Like
Ive got an x99 in an alien gear case with 64gb DDR4 and 1500watt PSU. I’ve got a GTX Titan X 12gb gpu. I can grab another GTX Titan X for pretty damn cheap. My thinking is that I can run one of these capable QWEN models on 24gb of GPUs and 64gb RAM using Strata or XeStrata. Anyone experience mis match issues between Pascal and Maxwell cards? CUDA version limitations? I am still uber new to all this and trying to make sure I don’t waste to much time, energy, or money trying to catch this wave. I’m also terrified of poisoned packages and models, because If I was vibe coding malware that seems like a nice target full of low information enthusiasts like myself. I was all jazzed to start implementing Tailscale…….. Seems like its the wild west again out here. Hard to know whats safe to play with and how.