are you here?
Aww, jeez, I missed this thread.
Sometimes things slip under my radar, or I get distracted. Lots of my posts occur while at work.
As far as those errors go, I've never seen them before. I'm going to do some research and get back to you.
Based on the formatting of the text, I'm guessing that this is output from kernel log?
2 Likes
the output of : dmesg | grep DMAR
Perfect! I've got a potential solution for you.
Have a read through this thread. If this seems like the situation you're in, try the solution suggested in the article.
To document the solution here in case the link dies, change intel_iommu=on in the grub config to intel_iommu=pt. This disables DMAR but still allows KVM to access the IOMMU and allows for benefit of interrupt remapping.
2 Likes
Now that I've got my GPU passthrough working smoothly, I feel more able to help.
I've switched from Ubuntu to OpenSUSE Tumbleweed for passthrough purposes, and it was a breeze.
I know this is what you want to avoid, but it may be worth trying Fedora (This method of passthrough is software from Red Hat, after all) or OpenSUSE. Ubuntu 17.04 worked for me out of the box, but I can't say why if it isn't working for other people.
EDIT: Of course, try the solution from @SgtAwesomesauce first so you don't have to hop if you can avoid it. 
2 Likes
OpenSUSE is one of those that doesn't quite feel right for me, and I can't pinpoint it. It's a wonderful server OS, but just doesn't feel right on the desktop. Just my 2c.
The issue @tatsu is encountering is not the result of Ubuntu having problems, it's the result of lack of hardware support in the mainline kernel. This has no solution as far as I know and if OP is using the latest kernel, there's really nowhere we can go from there.
I'm not using the lastest possible Kernel however thing are looking better now :
[ 0.000000] ACPI: DMAR 0x000000008D2383E8 0000BC (v01 A M I OEMDMAR 00000001 INTL 00000001)
[ 0.086589] DMAR: Host address width 46
[ 0.086590] DMAR: DRHD base: 0x000000fbffc000 flags: 0x1
[ 0.086594] DMAR: dmar0: reg_base_addr fbffc000 ver 1:0 cap d2078c106f0462 ecap f020fe
[ 0.086594] DMAR: RMRR base: 0x0000008cedb000 end: 0x0000008cf05fff
[ 0.086595] DMAR: ATSR flags: 0x0
[ 0.086596] DMAR: RHSA base: 0x000000fbffc000 proximity domain: 0x0
[ 0.086598] DMAR-IR: IOAPIC id 0 under DRHD base 0xfbffc000 IOMMU 0
[ 0.086598] DMAR-IR: IOAPIC id 2 under DRHD base 0xfbffc000 IOMMU 0
[ 0.086599] DMAR-IR: HPET id 0 under DRHD base 0xfbffc000
[ 0.086600] DMAR-IR: Queued invalidation will be enabled to support x2apic and Intr-remapping.
[ 0.086823] DMAR-IR: Enabled IRQ remapping in x2apic modeand
t@ts:~$ uname -a
Linux ts 4.10.0-20-generic #22-Ubuntu SMP Thu Apr 20 09:22:42 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
2 Likes
Well, 4.10 is recent enough. You can look at patch notes for minor releases and you'll be able to tell if it adds the support you need, which it doesn't.
If you run into additional trouble, let me know!
1 Like
IOMMU Group * 00:00.0 Host bridge [0600]: Intel Corporation Xeon E5/Core i7 DMI2 [8086:3c00] (rev 07)
00:01.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 1a [8086:3c02] (rev 07)
00:01.1 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 1b [8086:3c03] (rev 07)
00:02.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 2a [8086:3c04] (rev 07)
00:03.0 PCI bridge [0604]: Intel Corporation Xeon E5/Core i7 IIO PCI Express Root Port 3a in PCI Express Mode
[8086:3c08] (rev 07)
00:05.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Address Map, VTd_Misc, System Management
[8086:3c28] (rev 07)
00:05.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Control Status and Global Errors [8086:3c2a] (rev 07)
00:05.4 PIC [0800]: Intel Corporation Xeon E5/Core i7 I/O APIC [8086:3c2c] (rev 07)
00:11.0 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Virtual Root Port [8086:1d3e] (rev 05)
00:16.0 Communication controller [0780]: Intel Corporation C600/X79 series chipset MEI Controller #1 [8086:1d3a] (rev 05)
00:1a.0 USB controller [0c03]: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #2 [8086:1d2d] (rev 05)
00:1b.0 Audio device [0403]: Intel Corporation C600/X79 series chipset High Definition Audio Controller [8086:1d20] (rev 05)
00:1c.0 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 1 [8086:1d10] (rev b5)
00:1c.1 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 2 [8086:1d12] (rev b5)
00:1c.2 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 3 [8086:1d14] (rev b5)
00:1c.3 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 4 [8086:1d16] (rev b5)
00:1c.5 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 6 [8086:1d1a] (rev b5)
00:1c.6 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 7 [8086:1d1c] (rev b5)
00:1c.7 PCI bridge [0604]: Intel Corporation C600/X79 series chipset PCI Express Root Port 8 [8086:1d1e] (rev b5)
00:1d.0 USB controller [0c03]: Intel Corporation C600/X79 series chipset USB2 Enhanced Host Controller #1
[8086:1d26] (rev 05)
00:1e.0 PCI bridge [0604]: Intel Corporation 82801 PCI Bridge [8086:244e] (rev a5)
00:1f.0 ISA bridge [0601]: Intel Corporation C600/X79 series chipset LPC Controller [8086:1d41] (rev 05)
00:1f.2 SATA controller [0106]: Intel Corporation C600/X79 series chipset 6-Port SATA AHCI Controller [8086:1d02] (rev 05)
00:1f.3 SMBus [0c05]: Intel Corporation C600/X79 series chipset SMBus Host Controller [8086:1d22] (rev 05)
03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480]
[1002:67df] (rev c7)
03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Device [1002:aaf0]
04:00.0 VGA compatible controller [0300]: NVIDIA Corporation GK104 [GeForce GTX 760] [10de:1187] (rev a1)
04:00.1 Audio device [0403]: NVIDIA Corporation GK104 HDMI Audio Controller [10de:0e0a] (rev a1)
07:00.0 USB controller [0c03]: Texas Instruments TUSB73x0 SuperSpeed USB 3.0 xHCI Host Controller [104c:8241] (rev 02)
08:00.0 USB controller [0c03]: ASMedia Technology Inc. ASM1042 SuperSpeed USB Host Controller [1b21:1042]
09:00.0 Ethernet controller [0200]: Broadcom Limited NetLink BCM57781 Gigabit Ethernet PCIe [14e4:16b1] (rev 10)
0a:00.0 FireWire (IEEE 1394) [0c00]: VIA Technologies, Inc. VT6315 Series Firewire Controller [1106:3403] (rev 01)
0b:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
0c:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 01)
ff:08.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link 0 [8086:3c80] (rev 07)
ff:08.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link Reut 0 [8086:3c83] (rev 07)
ff:08.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link Reut 0 [8086:3c84] (rev 07)
ff:09.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link 1 [8086:3c90] (rev 07)
ff:09.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link Reut 1 [8086:3c93] (rev 07)
ff:09.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 QPI Link Reut 1 [8086:3c94] (rev 07)
ff:0a.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Power Control Unit 0 [8086:3cc0] (rev 07)
ff:0a.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Power Control Unit 1 [8086:3cc1] (rev 07)
ff:0a.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Power Control Unit 2 [8086:3cc2] (rev 07)
ff:0a.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Power Control Unit 3 [8086:3cd0] (rev 07)
ff:0b.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Interrupt Control Registers [8086:3ce0] (rev 07)
ff:0b.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Semaphore and Scratchpad Configuration
Registers [8086:3ce3] (rev 07)
ff:0c.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 07)
ff:0c.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 07)
ff:0c.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 07)
ff:0c.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller System Address
Decoder 0 [8086:3cf4] (rev 07)
ff:0c.7 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 System Address Decoder [8086:3cf6] (rev 07)
ff:0d.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 07)
ff:0d.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 07)
ff:0d.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Unicast Register 0 [8086:3ce8] (rev 07)
ff:0d.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller System Address
Decoder 1 [8086:3cf5] (rev 07)
ff:0e.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Processor Home Agent [8086:3ca0] (rev 07)
ff:0e.1 Performance counters [1101]: Intel Corporation Xeon E5/Core i7 Processor Home Agent Performance Monitoring [8086:3c46] (rev 07)
ff:0f.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Registers
[8086:3ca8] (rev 07)
ff:0f.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller RAS Registers
[8086:3c71] (rev 07)
ff:0f.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 0 [8086:3caa] (rev 07)
ff:0f.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 1 [8086:3cab] (rev 07)
ff:0f.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 2 [8086:3cac] (rev 07)
ff:0f.5 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 3 [8086:3cad] (rev 07)
ff:0f.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Target Address Decoder 4 [8086:3cae] (rev 07)
ff:10.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 0 [8086:3cb0] (rev 07)
ff:10.1 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 1 [8086:3cb1] (rev 07)
ff:10.2 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 0
[8086:3cb2] (rev 07)
ff:10.3 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 1
[8086:3cb3] (rev 07)
ff:10.4 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3
Thermal Control 2 [8086:3cb4] (rev 07)
ff:10.5 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller Channel 0-3 Thermal Control 3 [8086:3cb5] (rev 07)
ff:10.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 2 [8086:3cb6] (rev 07)
ff:10.7 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Integrated Memory Controller ERROR Registers 3 [8086:3cb7] (rev 07)
ff:11.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 DDRIO [8086:3cb8] (rev 07)
ff:13.0 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 R2PCIe [8086:3ce4] (rev 07)
ff:13.1 Performance counters [1101]: Intel Corporation Xeon E5/Core i7 Ring to PCI Express Performance Monitor
[8086:3c43] (rev 07)
ff:13.4 Performance counters [1101]: Intel Corporation Xeon E5/Core i7 QuickPath Interconnect Agent Ring Registers
[8086:3ce6] (rev 07)
ff:13.5 Performance counters [1101]: Intel Corporation Xeon E5/Core i7 Ring to QuickPath Interconnect Link 0
Performance Monitor [8086:3c44] (rev 07)
ff:13.6 System peripheral [0880]: Intel Corporation Xeon E5/Core i7 Ring to QuickPath Interconnect Link 1
Performance Monitor [8086:3c45] (rev 07)I don't understand why the IOMMU group script print out something entirely different for me.
plus wasn't DMAR: IOMMU enabled supposed to show up in dmesg | grep DMAR? is it because my kernel is too low?
So, first thing's first, we changed your system to passthrough, rather than full iommu with groups, so we're disabling DMAR entirely.
That is also why your IOMMU groups are different, if I remember correctly. Go ahead and skip the verification steps where you check your IOMMU groups.
Also, when I created the guide, Linux 4.6 was the latest release.
1 Like
given the context (that I'm blacklisting an NVIDIA gpu to passtrough and host is AMD) is this normal? :
t@ts:/etc/modprobe.d$ ls
alsa-base.conf blacklist.conf blacklist-framebuffer.conf blacklist-oss.conf intel-microcode-blacklist.conf
mlx4.conf
blacklist-ath_pci.conf blacklist-firewire.conf blacklist-modem.conf blacklist-rare-network.conf iwlwifi.confdo I now need to create nouveau.conf ?
Yes and Yes.
Ubuntu does some module configuration by default. That's why there are existing config files.
If I don't specifically tell you not to in the guide, you'll need to create any files that aren't there.
1 Like
Thank you! another issue (I must be an issue magnet ) :
t@ts:~$ ls /etc/dracut.conf
ls: cannot access '/etc/dracut.conf': No such file or directory
t@ts:~$ ls /etc/mkinitcpio.conf
ls: cannot access '/etc/mkinitcpio.conf': No such file or directory
t@ts:~$ whereis dracut
dracut:
t@ts:~$ whereis mkinitcpio
mkinitcpio:
t@ts:~$I guess I should install one? but which one?
Damn. I thought dracut was the function for initrd on Ubuntu. @DoctorDemand, did you rebuild your initrd when you were on Ubuntu? I'm not an Ubuntu pro.
/etc/mkinitcpio.conf is an Arch thing.
Let me see what I can find.
Not so much of an issue magnet, more of the guinea pig for Ubuntu. I never really tested the guide on it, so there's that.
1 Like
initrd/dracut is OpenSUSE,
On Ubuntu its update-initramfs -u I believe.
1 Like
Aaahhhh. That's what it was. They were using dracut a few years back if I remember correctly.
1 Like
Yeah, I think they were in 14.04.
1 Like
@tatsu, have a look at /etc/initramfs-tools/modules and see if that looks right.
I don't have access to Ubuntu at the moment, so I can't confirm, but all you need to do for this config is add the kernel modules to the list then run update-initramfs -u and you should be good.
1 Like
yes this thread confirms this is the way (plus lets me know syntax) :
1 Like
oh I almost forgot : I didn't blacklist anything more than THE vga port. why is it you also blacklist a usb and the nvidia audio? will driver install in the VM fail if I don't?