I am planning a build with the new 10th gen Intel processors and the Z490 chipset. I would like to be able to setup as many as 3 Windows guests, meaning up to 3 GPUs passed through to guest VMs running Windows. I know that AMD Ryzen 3xxx and TR would likely be better for this with the extra cores (for both Ryzen and TR) and PCIe lanes on TR, but for other reasons, I am going Intel on this one.
First question: My understanding is that with integrated graphics, e.g. i7-10700K with UHD 630, I can use that for the Linux host, and as long as I have 3 PCIe x16 slots (and they are each in their own IOMMU group), I can pass each of them to individual Windows guest VMs. Is that correct?
Second question: Ideally, the PCIe lanes used for the GPUs should be attached to the CPU, to avoid the bottleneck of having to go through the DMI from the chipset, right?
Third question: (assuming answer to question #2 is “yes”) Looking at the Z490 motherboards that will be available with 3+ PCIe x16 slots connected to CPU, I see the Asus ROG STRIX Z490-H GAMING and Supermicro C9Z490-PG. Based on the spec sheets, I believe the Asus is bifurcating the CPU lanes into x8, x4, and x4, whereas the Supermicro is using a PLX chip to multiplex into x8, x8, x8, x8. Any thoughts as to which board would work better for my application? Is it more difficult to passthrough with a PLX chip? If x4 is enough for GPU, then I could avoid the extra cost of the PLX chip.
I don’t know how bad it is to pass GPU stream through plex chip but as far as I know it’s virtually passing your GPU through chipset. As in Plex chip adds latency while bifurcation is just lane splitting so there are no compnents in between. For gaming it’s always recommended to use direct lanes and not chipset ones to avoid said latency and bottlenecks. Also Plex chips don’t magically add bandwidth. You will still get the same lane bandwidth but the beauty of Plex chip is that if say one graphics card idles then others will get more bandwidth available while bifurcation doesn’t do that. Though like I said I don’t know how bad plex chip latency is and also depends on how good of Plex chip used, some have low latency some have high. I would think that if Asus and other “gaming” mobo manufactures didn’t see plex chip as bad thing they would have used one long time ago at least in their high tier mobos. Server manufacturer obviously doesn’t care about latency, it’s all about bandwidth and expansions.
yeah, what did you end up building? I’m looking at the C9Z490-PG too, mainly because its included 10Gb LAN and extra slots. I’m only planning on running 1 Win VM with a GPU passed through. Other VM’s are fine without any GPU for my use.
But I kind of have the same question about that PLX chip. For me, I’ll be playing Elite Dangerous and the new Flight Simulator. Nothing needing high FPS. I’m wondering if I’d even notice a difference passing in a good GPU at lower FPS. (in the 30-60 range should be happy for me)
The extra slots I don’t really need, but I like options for random weird things with VM’s or ZFS stuff with SAS controllers. I may throw in a GT750 card for testing vm graphics, or just better graphics and direct monitor use.
FYI I’ve been in touch with their tech support at Supermicro. The C9Z490-PG board puts all x16 PCIe slots into a single IOMMU Group due to the PLX chip. That’s a huge bummer. It would almost be perfect for me if not for that.
The 1x slot is in its own group, as are the m.2 (each). That effectively makes this board useful for one VM to passthrough and no other host cards for SAS or other 8x GPU’s. The 10G LAN would be very handy. Also, according to their manual, all of the PCIe slots use 16 lanes, if I understand that correctly. So even for gaming and no VFIO use you’d be limited to a single GPU and no other cards. (whats the point of the 4 slots then?)