4× RTX PRO 6000 Blackwell WS — PCIe P2P/NCCL compatibility: Xeon 676X vs Threadripper PRO 9975WX vs EPYC 9005?

Have done all of this with both w790 and epyc 9005 turin setups. if ur going serious about a 4+ gpu setup consider the bios features as really key, the ability to enumerate 8+ GPUs is not common, people on the threadripper boards in discord struggle all the time to get past 7 having to disable features like usb4.

consider where a box like this will run. a pc on your desk this aint, the power alone of (in my case 5x1600w PSUs) relegates it to a rack in the basement. The Rig 3.0 for pictures.

p2p works great even with consumer cards and the right drivers. bios-grub-kernel settings are what is make or break. pcie signal quality is really important if you are going to be beating the piss out of it. these things can and will each suck 100GB/s bidirectionally, pcie is not enough for high end large model work and will bottleneck constantly. Consider a PCIE switch miwin and c-payne are most reputable sources.

find me on discord with questions, or come to the blackwell performance community hardware channels. there are dedicated pcie switching discussions with tons of pics stories and setups.

3 Likes