Hello everyone. I’m new here and new to running AI and LLMs on a home rig. I have a somewhat unique setup in that I’m trying to run an LLM on two external GPUs. I have a Minisforum MS-02 Ultra with 256 GB DDR5 system memory. The computer has 2 Thunderbolt 4 plus ports, but they are run by a single Intel controller. I have two Aorus RTX 5090 AI Boxes that I am trying to run together to get 64 GB of VRAM to run larger LLMs on. Both units are recognized by the computer and show up in the Task Manager. I’m very new to running AI models but so far I’ve started working with ComfyUI and yesterday was able to load my first model (qwen 3.8 27B) in LM Studio. I’m absolutely amazed at what this can do and to think that it’s actually running on my own home PC just blows me away. My problem is this, I want to run bigger models but when I try to load a MOE model that’s supposed to work across two GPUs, it fails to load. Any suggestions on what I might be doing wrong or is there a specific model that anyone could recommend? I know there is a latency issue with the 2 Thunderbolt 4+ connections splitting a single PCIE 5 lane, but that seems to only be an issue when loading a model. Once loaded, I get pretty decent performance in ComfyUI and when running a chatbot in qwen 3.8 27B. Any help would be greatly appreciated. - CJ
You didn’t say, but I’m assuming you are running Windows 11? I have certainly run into issues with multiple GPU setups where the GPUs are in direct to CPU PCIe slots, mostly all in Win 11 so far, I have been running Cachey OS more recently on the 2 multi GPU rigs and I haven’t run into much of any real issue so far. You may be hitting operating system problems/ limits.
You can also try loading a much smaller model that should fit in 1 or 2 GPUs after ensuring both GPUs are showing up and selected in LM Studio and that “split model evenly” is selected (under the hardware tab). If you can run smaller models across 2 GPUs loaded with context only into the VRAM, it should help narrow down what is happening.
I’ve also had better luck getting some models to load & run fast with minimal headache or having to fiddle around by using the Unsloth Desktop program. It’s similar to LM Studio, but in several ways seems better (probably because LMStudio was focusing on and is now pushing Bionic)
To my knowledge ComfyUI and image/ audio/ video generation cannot utilize more than 1 GPU in a single instance.
You are correct. Windows 11 Pro. Thanks for the tips. I didn’t think to see if I could split a smaller model first. I did select the “split model evenly” in LM Studio but to be honest, I’m still trying to figure everything out. Chat GPT has been pretty good so far helping me to get started. I communicated with Minisforum and they said that I shouldn’t have a problem running two EGPUS. I did have to make some adjustments to the bios before Windows would recognize both units. Thanks again for your suggestions. - CJ