Ubuntu 18.04 LTS ACS patch problem

Software & Operating Systems Linux
1

Hey guys,

I’m having some difficulties splitting devices on my Asrock Fatal1ty X370 Professional Gaming.

After updating the BIOS I have these IOMMU groups:

IOMMU Group 0 00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 10 00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 11 00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]
IOMMU Group 12 00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 59)
IOMMU Group 12 00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
IOMMU Group 13 00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 0 [1022:1460]
IOMMU Group 13 00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 1 [1022:1461]
IOMMU Group 13 00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 2 [1022:1462]
IOMMU Group 13 00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 3 [1022:1463]
IOMMU Group 13 00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 4 [1022:1464]
IOMMU Group 13 00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 5 [1022:1465]
IOMMU Group 13 00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric Device 18h Function 6 [1022:1466]
IOMMU Group 13 00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Data Fabric: Device 18h; Function 7 [1022:1467]
IOMMU Group 14 01:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM951/PM951 [144d:a802] (rev 01)
IOMMU Group 15 03:00.0 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b9] (rev 02)
IOMMU Group 15 03:00.1 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b5] (rev 02)
IOMMU Group 15 03:00.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:43b0] (rev 02)
IOMMU Group 15 1d:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 15 1d:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 15 1d:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 15 1d:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] 300 Series Chipset PCIe Port [1022:43b4] (rev 02)
IOMMU Group 15 1e:00.0 Ethernet controller [0200]: Aquantia Corp. AQC108 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:d108] (rev 02)
IOMMU Group 15 20:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 15 21:00.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 15 26:01.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 15 26:03.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 15 26:05.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 15 26:07.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 15 27:00.0 Network controller [0280]: Intel Corporation Device [8086:24fb] (rev 10)
IOMMU Group 15 2a:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU Group 16 2d:00.0 VGA compatible controller [0300]: NVIDIA Corporation GM204 [GeForce GTX 970] [10de:13c2] (rev a1)
IOMMU Group 16 2d:00.1 Audio device [0403]: NVIDIA Corporation GM204 High Definition Audio Controller [10de:0fbb] (rev a1)
IOMMU Group 17 2e:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] [10de:1b06] (rev a1)
IOMMU Group 17 2e:00.1 Audio device [0403]: NVIDIA Corporation GP102 HDMI Audio Controller [10de:10ef] (rev a1)
IOMMU Group 18 2f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device [1022:145a]
IOMMU Group 19 2f:00.2 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Platform Security Processor [1022:1456]
IOMMU Group 1 00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 20 2f:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller [1022:145c]
IOMMU Group 21 30:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Device [1022:1455]
IOMMU Group 22 30:00.2 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
IOMMU Group 23 30:00.3 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) HD Audio Controller [1022:1457]
IOMMU Group 2 00:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 3 00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 4 00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 5 00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 6 00:03.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe GPP Bridge [1022:1453]
IOMMU Group 7 00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 8 00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) PCIe Dummy Host Bridge [1022:1452]
IOMMU Group 9 00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) Internal PCIe GPP Bridge 0 to Bus B [1022:1454]

It’s not bad, GPUs have their own groups, but I want to split the 2 LAN ports into separate groups (one to be passed to my Windows VM)

I have compiled the latest kernel from kernel.org (at the time of writing it’s 4.17-rc5), applied the patch from this guide from GrayBoltWolf with the first part of the patch (documentation part) removed.

Installed the compiled .deb files, added “pcie_acs_override=downstream,multifunction” to the grub file, ran upate-grub, reboot and…same groups. Nothing has changed.

Weird thing is that on Ubuntu 16.04 LTS the acs patch did split the IOMMU groups further…and I don’t wanna go back now.

Any help is appreciated.

PS: Not sure I chose the right category.

2

Which ACS patch did you use? From what I understand/remember, there are slight differences in the acs patch from kernel revision to kernel. Did the patch apply without error?

Edited to add, did you also put amd_iommu=on in the default grub command line?

3

I used the one from Wolf’s guide. This is the link: https://drive.google.com/open?id=0B7qwHfGFYCIIcGxEcHc5X3l0R1k

After removing the documentation part, it was applied succesfully.

4

GRUB_DEFAULT=0
GRUB_HIDDEN_TIMEOUT=0
GRUB_HIDDEN_TIMEOUT_QUIET=true
GRUB_TIMEOUT=0
GRUB_DISTRIBUTOR=lsb_release -i -s 2> /dev/null || echo Debian
GRUB_CMDLINE_LINUX_DEFAULT=“amd_iommu=on iommu=1 pcie_acs_override=downstream,multifunction”
GRUB_CMDLINE_LINUX=""

This is my grub file

5

I’m pretty sure the patch you are using is for a much older kernel. This is the one I’m using for 4.14-* differs from yours

https://pastebin.com/58enjHAE

6

yeah, I just found it. It changes again for 4.16.*

https://aur.archlinux.org/cgit/aur.git/tree/add-acs-overrides.patch?h=linux-vfio

7

tried both. the one from pastebin and the one from arch linux. still the same grouping. :frowning:

8

Did you enable (not auto) IOMMU in bios and also enable virtualization? Did you remember to run “sudo update-initramfs -u”? To get basic ACS working in Ubuntu 18.04 on AMD systems, there’s really only 2 files you have to change: /etc/default/grub (which you’ve done) and /etc/modprobe.d/vfio-pci.conf which contains (in my example):

#IOMMU Group 30 0c:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [GeForce GTX 1070] [10de:1b81] (rev a1)
#IOMMU Group 31 0c:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
#IOMMU Group 27 09:04.0 Multimedia audio controller [0401]: C-Media Electronics Inc CMI8788 [Oxygen HD Audio] [13f6:8788]
#IOMMU Group 26 07:00.0 Ethernet controller [0200]: Intel Corporation 82574L Gigabit Network Connection [8086:10d3]
#IOMMU Group 34 0d:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Family 17h (Models 00h-0fh) USB 3.0 Host Controller [1022:145c]
#IOMMU Group 25 06:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9235 PCIe 2.0 x2 4-port SATA 6 Gb/s Controller [1b4b:9235] (rev 11)

options vfio-pci ids=10de:1b81,10de:10f0,1022:145c,8086:10d3,13f6:8788,1b4b:9235

I comment out the groups so I remember what those ids mean in the future. Nothing has to go in /etc/modules The guide over-complicates a few things. If you want to passthrough a video card in a 2 card system (im passing through NV with an AMD graphics card as the primary) the only thing you may need to do is /etc/modprobe.d/nvidia.conf which contains:

softdep nouveau pre: vfio vfio_pci

EDITED to add:
You might need this (with the id’s configured to your system of course) in /etc/initramfs-tools/modules:
vfio
vfio_iommu_type1
vfio_virqfd
vfio_pci
vhost-net

vfio-pci ids=1b21:0612,8086:10d6

EDIT #2 The above I have on my server box for some reason, I don’t have it on my main box with runs a win10 KVM guest. I don’t have anything in /etc/initramfs-tools/modules and it still works, so I dunno if its really needed at all.

9

Yes, I did all that. GPU passthrough is not an issue. Just splitting the group that has the two LAN controllers.

I feel like this Ubuntu 18.04 is the issue since it worked on 16.04, which is very fustrating.

10

I have Ubuntu 18.04 on both my desktop and server, both have the ACS patch applied. Maybe it’s the kernel you’re trying to use? Try the stable branch kernel, 4.14.* is the best suggestion I have left.

11

With any kernel below 4.16.* I get a kernel panic when I want to reboot or shutdown the PC :frowning: I have the 1800x.

12

Hmm… Interesting. My server has a 2700x on a 370x Taichi, and my desktop has an 1800x on a Asus Crosshair Hero IV

Im running kernel 4.14.39 with the acs patch

13

Having a similar issue here. Running 18.04 with the kernel 4.20 and the ACS patch, trying to isolate my USB 3.1 controller to pass it through for hotplugging. Doesn’t seem like it wants to work.