I’m looking to pull the trigger on a Quadro M4000 or M2000 to drive the monitors on my linux machine and pass through the existing GTX 970 to VMs to play games. My motherboard is a Gigabyte H170M-D3H [1] which has a PCIE x16 and another PCIE x16 that actually only runs as a x4. Here are my questions:
Is running a Quadro on linux a good idea if all I want to do is power a few 4k monitors and run some video and web browsing on them? (No heavy 3D required) How is Quadro support on Linux for the M4000 and M2000? I can’t seem to get a straight answer.
How can I tell my computer to use the Quadro to boot on? I can only find on-board enable/disable options in the BIOS. What happens if I have multiple PCI graphics cards? Don’t forget I will have a dummy EDID dongle attached. I want to be able to see the grub screen.
Will running the Quadro on a x4 PCIE slot be a noticeable bottleneck for desktop apps? I don’t think so, but just wanted to be sure.
Is there some reason my motherboard or GTX card be a problem? PCI reset problems? I couldn’t find anything on the looking glass website, but I worry I may have missed something.
Thank you.
[1] - Sorry, I can’t include links in my posts. Please google the model if needed.
High quality motherboards (at least high quality for what I want) should have an option to switch which graphics output it should boot too.
Other motherboards (that I am aware of) default to booting on the top PCIe slot that has a graphics card.
Should be just fine for normal desktop apps.
However, I actually do not know if that is fine for Looking Glass in all cases.
That PCIe slot is probably feed through the DMI and chipset, not direct to the CPU, so the bandwidth if shared with basically all the other devices the motherboard has, so other pcie, nvme, network, usb, etc.
I expect it would be fine, I just do not know for certain.
There should be no issues with the 970.
The thing to look for on the motherboard is what the IOMMU groups look like.
High quality motherboards (at least high quality for what I want) should have an option to switch which graphics output it should boot too.
Looked again and found it! Problem solved!
The thing to look for on the motherboard is what the IOMMU groups look like.
I enabled IOMMU support in the kernel and got the following groups:
IOMMU Group 0:
00:00.0 Host bridge [0600]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor Host Bridge/DRAM Registers [8086:191f] (rev 07)
IOMMU Group 1:
00:01.0 PCI bridge [0604]: Intel Corporation 6th-9th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
01:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
IOMMU Group 2:
00:14.0 USB controller [0c03]: Intel Corporation 100 Series/C230 Series Chipset Family USB 3.0 xHCI Controller [8086:a12f] (rev 31)
IOMMU Group 3:
00:16.0 Communication controller [0780]: Intel Corporation 100 Series/C230 Series Chipset Family MEI Controller #1 [8086:a13a] (rev 31)
IOMMU Group 4:
00:17.0 SATA controller [0106]: Intel Corporation Q170/Q150/B150/H170/H110/Z170/CM236 Chipset SATA Controller [AHCI Mode] [8086:a102] (rev 31)
IOMMU Group 5:
00:1b.0 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #19 [8086:a169] (rev f1)
IOMMU Group 6:
00:1b.3 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #20 [8086:a16a] (rev f1)
IOMMU Group 7:
00:1c.0 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #3 [8086:a112] (rev f1)
IOMMU Group 8:
00:1c.4 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #5 [8086:a114] (rev f1)
IOMMU Group 9:
00:1d.0 PCI bridge [0604]: Intel Corporation 100 Series/C230 Series Chipset Family PCI Express Root Port #9 [8086:a118] (rev f1)
IOMMU Group 10:
00:1f.0 ISA bridge [0601]: Intel Corporation H170 Chipset LPC/eSPI Controller [8086:a144] (rev 31)
00:1f.2 Memory controller [0580]: Intel Corporation 100 Series/C230 Series Chipset Family Power Management Controller [8086:a121] (rev 31)
00:1f.3 Audio device [0403]: Intel Corporation 100 Series/C230 Series Chipset Family HD Audio Controller [8086:a170] (rev 31)
00:1f.4 SMBus [0c05]: Intel Corporation 100 Series/C230 Series Chipset Family SMBus [8086:a123] (rev 31)
IOMMU Group 11:
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8] (rev 31)
IOMMU Group 12:
02:00.0 PCI bridge [0604]: ASMedia Technology Inc. ASM1083/1085 PCIe to PCI Bridge [1b21:1080] (rev 04)
IOMMU Group 13:
07:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
The GTX 970 will be for the guest, so I think the fact that it isn’t completely isolated isn’t a problem because the guest can also get the 00:01.0 device. (Right?)
I wish I knew if I bought a Quadro M2000 or M4000 and plugged it into the other PCIE x16butreallyx4 slot which group it would end up in. I don’t really have a PCI device to test. Can anybody tell just by looking at this output? I don’t have a block diagram, but I do have a layout.
It is actually fine to not pass through the other device in this one case, as the other device in the group is a PCIe controller. Basically, just ignore it.
I would not be shocked if it ends up with the nvme drive or is in it’s own group.
Either way, it does not matter which group the Quadro would go in, because it is not the card that is getting passed through.
Generally, the block diagram is more or less shared between all motherboards that use a chipset, so in your case it is H170. Sometimes it can vary depending on what CPU family is used (for example AMD apus have less PCIe lanes available than AMD CPUs do).
