Thanks for the response. That is all interesting info that I’ll keep in mind, but I think you misunderstand what I’m talking about. I want to pass the PCIe card into a Windows virtual machine using IOMMU. That’s what I mean by passthrough, not the AV signal.
20 seconds, I’d give you, depending on hardware and memory usage, but 1 minute? no way.
This is a problem.
Let’s try to fix the other two issues first.
I’ll need a bit of information.
iommu.sh output, please.
lspci -knn output as well.
PC specs, including motherboard model and firmware version.
Are you passing through any other PCI devices?
Output of iommu.sh:
IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers [8086:3ec2] (rev 07)
IOMMU Group 10 00:1d.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #9 [8086:a298] (rev f0)
IOMMU Group 11 00:1f.0 ISA bridge [0601]: Intel Corporation Z370 Chipset LPC/eSPI Controller [8086:a2c9]
IOMMU Group 11 00:1f.2 Memory controller [0580]: Intel Corporation 200 Series/Z370 Chipset Family Power Management Controller [8086:a2a1]
IOMMU Group 11 00:1f.3 Audio device [0403]: Intel Corporation 200 Series PCH HD Audio [8086:a2f0]
IOMMU Group 11 00:1f.4 SMBus [0c05]: Intel Corporation 200 Series/Z370 Chipset Family SMBus Controller [8086:a2a3]
IOMMU Group 12 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8]
IOMMU Group 13 02:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
IOMMU Group 14 04:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller [1b21:2142]
IOMMU Group 15 05:00.0 Multimedia controller [0480]: YUAN High-Tech Development Co., Ltd. Device [12ab:0380]
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 980] [10de:13c0] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
IOMMU Group 2 00:02.0 Display controller [0380]: Intel Corporation UHD Graphics 630 (Desktop) [8086:3e92]
IOMMU Group 3 00:14.0 USB controller [0c03]: Intel Corporation 200 Series/Z370 Chipset Family USB 3.0 xHCI Controller [8086:a2af]
IOMMU Group 4 00:16.0 Communication controller [0780]: Intel Corporation 200 Series PCH CSME HECI #1 [8086:a2ba]
IOMMU Group 5 00:17.0 RAID bus controller [0104]: Intel Corporation SATA Controller [RAID mode] [8086:2822]
IOMMU Group 6 00:1b.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #17 [8086:a2e7] (rev f0)
IOMMU Group 7 00:1c.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #1 [8086:a290] (rev f0)
IOMMU Group 8 00:1c.4 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #5 [8086:a294] (rev f0)
IOMMU Group 9 00:1c.7 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #8 [8086:a297] (rev f0)
Output of lspci -knn:
00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers [8086:3ec2] (rev 07)
Subsystem: ASUSTeK Computer Inc. PRIME H310M-D [1043:8694]
Kernel driver in use: skl_uncore
Kernel modules: ie31200_edac
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
Kernel driver in use: pcieport
00:02.0 Display controller [0380]: Intel Corporation UHD Graphics 630 (Desktop) [8086:3e92]
DeviceName: Onboard IGD
Subsystem: ASUSTeK Computer Inc. UHD Graphics 630 (Desktop) [1043:8694]
Kernel driver in use: i915
Kernel modules: i915
00:14.0 USB controller [0c03]: Intel Corporation 200 Series/Z370 Chipset Family USB 3.0 xHCI Controller [8086:a2af]
Subsystem: ASUSTeK Computer Inc. 200 Series/Z370 Chipset Family USB 3.0 xHCI Controller [1043:8694]
Kernel driver in use: xhci_hcd
00:16.0 Communication controller [0780]: Intel Corporation 200 Series PCH CSME HECI #1 [8086:a2ba]
Subsystem: ASUSTeK Computer Inc. 200 Series PCH CSME HECI [1043:8694]
Kernel driver in use: mei_me
Kernel modules: mei_me
00:17.0 RAID bus controller [0104]: Intel Corporation SATA Controller [RAID mode] [8086:2822]
Subsystem: ASUSTeK Computer Inc. SATA Controller [RAID mode] [1043:8694]
Kernel driver in use: ahci
Kernel modules: ahci
00:1b.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #17 [8086:a2e7] (rev f0)
Kernel driver in use: pcieport
00:1c.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #1 [8086:a290] (rev f0)
Kernel driver in use: pcieport
00:1c.4 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #5 [8086:a294] (rev f0)
Kernel driver in use: pcieport
00:1c.7 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #8 [8086:a297] (rev f0)
Kernel driver in use: pcieport
00:1d.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #9 [8086:a298] (rev f0)
Kernel driver in use: pcieport
00:1f.0 ISA bridge [0601]: Intel Corporation Z370 Chipset LPC/eSPI Controller [8086:a2c9]
Subsystem: ASUSTeK Computer Inc. Z370 Chipset LPC/eSPI Controller [1043:8694]
00:1f.2 Memory controller [0580]: Intel Corporation 200 Series/Z370 Chipset Family Power Management Controller [8086:a2a1]
Subsystem: ASUSTeK Computer Inc. 200 Series/Z370 Chipset Family Power Management Controller [1043:8694]
00:1f.3 Audio device [0403]: Intel Corporation 200 Series PCH HD Audio [8086:a2f0]
Subsystem: ASUSTeK Computer Inc. 200 Series PCH HD Audio [1043:8724]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
00:1f.4 SMBus [0c05]: Intel Corporation 200 Series/Z370 Chipset Family SMBus Controller [8086:a2a3]
Subsystem: ASUSTeK Computer Inc. 200 Series/Z370 Chipset Family SMBus Controller [1043:8694]
Kernel driver in use: i801_smbus
Kernel modules: i2c_i801
00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (2) I219-V [8086:15b8]
Subsystem: ASUSTeK Computer Inc. Ethernet Connection (2) I219-V [1043:8672]
Kernel driver in use: e1000e
Kernel modules: e1000e
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 980] [10de:13c0] (rev a1)
Subsystem: NVIDIA Corporation GM204 [GeForce GTX 980] [10de:1116]
Kernel driver in use: nvidia
Kernel modules: nvidiafb, nouveau, nvidia_drm, nvidia
01:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
Subsystem: NVIDIA Corporation GM204 High Definition Audio Controller [10de:1116]
Kernel driver in use: snd_hda_intel
Kernel modules: snd_hda_intel
02:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981/PM983 [144d:a808]
Subsystem: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 [144d:a801]
Kernel driver in use: nvme
Kernel modules: nvme
04:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM2142 USB 3.1 Host Controller [1b21:2142]
Subsystem: ASUSTeK Computer Inc. ASM2142 USB 3.1 Host Controller [1043:8756]
Kernel driver in use: xhci_hcd
05:00.0 Multimedia controller [0480]: YUAN High-Tech Development Co., Ltd. Device [12ab:0380]
Subsystem: Device [1cfa:0006]
Kernel driver in use: vfio-pci
PC Specs:
- Intel Core i5 8600k
- 16 GB DDR4 (I don’t know the specific DIMMs)
- Samsung 970 Evo 500GB NVMe drive
- ASUS PRIME Z370-A Motherboard (Version 1802)
- nVIDIA GeForce GTX 980 (Gigabyte GV-N980D5-4GD-B)
Output from dmesg | grep vfio-pci (found this in Googling similar issues):
[ 0.000000] Command line: initrd=\EFI\Pop_OS-3dbe593c-148b-49e3-a762-22a9cf82d27e\initrd.img root=UUID=3dbe593c-148b-49e3-a762-22a9cf82d27e ro quiet loglevel=0 systemd.show_status=false intel_iommu=on vfio-pci.disable_idle_d3=1 splash
[ 0.043216] Kernel command line: initrd=\EFI\Pop_OS-3dbe593c-148b-49e3-a762-22a9cf82d27e\initrd.img root=UUID=3dbe593c-148b-49e3-a762-22a9cf82d27e ro quiet loglevel=0 systemd.show_status=false intel_iommu=on vfio-pci.disable_idle_d3=1 splash
[ 146.498791] vfio-pci 0000:05:00.0: enabling device (0000 -> 0002)
[ 148.151887] vfio-pci 0000:05:00.0: not ready 1023ms after PM D3->D0; waiting
[ 149.244311] vfio-pci 0000:05:00.0: not ready 2047ms after PM D3->D0; waiting
[ 151.359228] vfio-pci 0000:05:00.0: not ready 4095ms after PM D3->D0; waiting
[ 155.709352] vfio-pci 0000:05:00.0: not ready 8191ms after PM D3->D0; waiting
[ 164.158996] vfio-pci 0000:05:00.0: not ready 16383ms after PM D3->D0; waiting
[ 181.569493] vfio-pci 0000:05:00.0: not ready 32767ms after PM D3->D0; waiting
[ 216.380406] vfio-pci 0000:05:00.0: not ready 65535ms after PM D3->D0; giving up
1 Like
Well then.
I suspect that’s where your slow startup comes up.
It can’t take it out of PCIe sleep mode.
Which would also explain why the windows VM can’t get any signal.
Had that same idea. In another thread @FurryJackman (?) mentioned that passthrough does not work with these cards though for some reason. Maybe he can elaborate on that.
I think (might be wrong) the Elgato capture thing relies on a GPU being present for encoding, because the cards do not have a hardware encoder.
Most likely they represent themselves as a generic USB Video Class.
3 Likes
It’s showing up as a PCI device which is not a USB controller. I doubt it.
He was talking about the Camlink though, which is USB, not PCI right 
Ah, well then that’s off topic.
1 Like
I’m very new to this, so please bear with me, but isn’t the “disable_idle_d3” option supposed to prevent it from going into D3 state? Is that option improperly applied? How can I verify that it is being applied to the card in Linux?
Yeah, see?!? That thing. … I totally nailed it! 
@trippsc2 If the fix “happens in hardware” also look at Magewell and AVIO. I have a Magewell 1080p HDMI to USB dongle and it works great everywhere. It also does more than the Elgato sticks so performance impact is gonna be minimal. AVIO was one of the first companies capable of 4k capture (at least for consumers). IIRC those also run on linux.
I would imagine so, let me have a look at the kernel docs.
I was thinking the PCI device, not the USB. 
1 Like
The HD60 Pro is a PCI device.
All good, was just a misunderstanding. 
1 Like
Looks like your disable_idle_d3 is properly defined.
Not really sure what’s going on.
1 Like
I’m not sure either. Thanks for looking at it!
You’re welcome.
I suspect, and I think we are going to have to wait for someone with more experience with the hardware to know for sure, that it’s simply incompatible with passthrough.
Yes, you can run UVC USB 3.0 capture cards in a VM but not PCI-E ones, since the PCI-E ones require FPGA firmware to be loaded on boot, and a virtual machine can’t do that.
Yes, NDI will work, but your quality goes down dramatically to the same level as ProRes 422 LT. (You will notice a lot of blocky artifacts in gradients.)
3 Likes
Yup, I understand, my reply was to people saying “use this other thing” which doesn’t solve your problem.
That makes sense. I’ll see what I can do about getting a bare metal streaming machine or USB capture card for what I’m trying to get done. Thanks for your help!
Bare metal Linux Magewell PCI-E system is best, as it supports drop frame framerates properly. Get a GPU with Turing NVENC and you’re all set.
1 Like