Hi all, first post on L1 so apologies if it’s in the wrong subforum.
I have a mATX PC with a single 5060Ti, and I’d like to add a second one. My intended use is for local LLM inference with a single model split over two GPUs. I’ve been exploring as many options as I can but keep running into roadblocks.
Here are my limitations:
Another GPU won’t fit in the case, nor into the second 16x slot of the motherboard.
My mobo, Asus TUF-GAMING B550M Plus, only has a single PCI-E 4.0 16x slot, which doesn’t support 8x/8x bifurcation in the BIOS*. The second 16x slot is 3.0, on chipset lanes, and I’ve ruled that out for bandwidth/latency reasons with LLM model splitting.
The mobo has a PCI-E 4.0 4x M.2 NVME slot on CPU lanes, but apparently the 5060ti doesn’t work well with m2 to pci-e extenders like ADT-Link’s K43SG because of … retiming issues? Given that the 5060ti is actually an 8x card with a 16x connector. [1][2]
Have a 550W Corsair PSU and Ryzen 5600X CPU at stock clocks/voltage. Happy to power-limit the GPU/s to 150W if that makes ‘em fit within the PSU’s operating capacity, or add a second PSU (eg Dell DA2).
Trying to do this in a budget friendly way!
I’d love some advice on what my best options are. Buy a new mobo and case? Try to do something with Oculink instead? Consider a different GPU? Any other routes I haven’t considered?
Thanks everyone.
* The Asus manual says the PCI16X_1 slot “Support PCIe bifurcation for RAID on CPU function.” implying that you can run 4 m2 SSDs into the main 16x slot with a 4x/4x/4x/4x splitter card. I can’t tell if that would work with an M2 to PCI-E splitter of some kind.
Your best bet, honestly, is to either upgrade your 5060 Ti to something with more VRAM, like the RX 9070…
Or find a high grade mATX motherboard combined with a 1 slot pro card. But yeah, given that good AM4 mobos are pretty much extinct now… If you can find a B550 Strix E ATX model it could.work.
I know that is not the advice you want to hear. But given that you are cornered, it is the best I can give.
Thanks for the response. The Strix E looks like a good option, because it has 2 x8 slots on the CPU. A new case and new motherboard would be about £200 used, it looks like.
You’ll need a motherboard with proper lane splitting support (x8 /x8 from the CPU). Otherwise the second GPU will run off the x4 chipset link and tensor parallel wouldn’t work or will be slow.
Better odds, would be involving a full-size ATX board, to get proper X8/X8 [IF adamant for (2) GPUs]
And since the newer __60s/60Ties, are all x8 connectivity– it could compliment, your current one
Which could mean doing over your pkg [should the setup, be explicitly mATX only limit]
Otherwise- you’re better off, working off a beefier GPU [+ that x4 lingering slot, could be used, for something else]
I’m not married to mATX! In fact I would be happy to go larger, to get more airflow and quieter operation. So I have set up an ebay alert for the Strix B550E and also a Strix X570E. Then I’ll probably look at a Fractal Define 7 or similar.
I measured my total system power draw at 250W peak during inference, with one 5060Ti at 180W. So if I dropped that to 150W it would be 220W, plus another 150W for a second 5060Ti, total of 370W. Does that sound OK for a Corsair RM550X PSU?
Dang, that’s an annoying thing from Asus. Maybe you could try your luck and ask for a firmware with x8/x8 bifurcation support, they were cases where they have supplied those for other motherboards. You’d still need a splitter anyway, which would incur costs and a janky way to have things set in your case.
I would not rule that out. For pipeline parallelism with something like a 5060ti it should not be a problem at all and the bandwidth would be more than enough. For tensor/graph parallelism is where you might see the bandwidth being a limiting factor, but I don’t think it’d be much of an issue with a 5060ti.
Another good mobo option that I can personally vouch for is the B550 ProArt. It did great for my 2x3090, 5950x and 128GB of RAM until I upgraded to AM5.
There’s also that AM4 spreadsheet, it’s a bit outdated but still useful:
Some of the X570 and B550 options that support x8/x8 on the two slots are: