Z370/Z390 Owners: What's your IOMMU?

So on intel, it seems like with the 6/8 core processors (8700, 8700k, 9900k, 9700k, etc) the IOMMU separation is pretty good for chipset devices, but for CPU devices there is no IOMMU separation.

With Z390, we’re seeing AIB partners offer x8/x4/x4 PCIe slots using CPU lanes, which is great for device flexibility and reducing bottlenecks, but all of those devices share a single IOMMU group which isn’t great for virtualization.

On the 7700k, at least some boards provided separation for x8/x8 devices, but that doesn’t really seem to be the case anymore (at least when using the latest UEFI).

For VFIO users who want to run a virtual machine, it is still possible to use the iGPU on Intel CPUs + one add-in card but for the user that wants to use two add-in cards you’d be relegated to another platform like Intel’s own X299 platform, or Ryzen/ Threadripper from AMD.

Generally, on higher end Xeon and X299 Intel CPUs, users of this virtualization have had an easier more stable time. And while AMD is quickly narrowing the gap, the high clock speed and performance of the 8-core 9900k may be appealing for certain types of users that would run this kind of virtualization.

Are you (or would you be) an Intel user that uses more than one IOMMU group from your CPU PCIe lanes? If so, let me know.

I’m hoping that there are enough of us to make it worth Intel’s while. Both MSI and Gigabyte have directly expressed interest in providing these features for users as well, which is extremely encouraging.

TLDR:

Please post :slight_smile:

Your Motherboard Model;
The output of ls-iommu.sh

1-2 sentences on what your intended use case is (multiple VMs, mixed host/VM add-in graphics, etc)

If you’ve purchased X299 in Lieu of 1151/Z370 for exactly this reason, I want to hear from you too.

thanks everyone!!!

8 Likes

Shameless bump.

Also requesting hdparm -Tt /dev/sd* if you’re running hot new nvmes. Bonus points if you’re willing to move them around or plug them to the PCIe slot via PCIe adapter.

As I want to know if anyone’s found it possible to run a z390 motherboard with:
16x PCIe video card
4x PCIe 10 Gb Ethernet card
4x PCIe raid controller

With at least the video card ending up in it’s own IOMMU Group.

Or did I hit the ‘Get x299’ response territory?

Hi Wendell,

A reply two months late, but hopefully better late than never.

MB: Gigabyte Auros Master Bios Rev: F7 w/Intel 9900K

Have very little experience with iommu, but wanted to send the output below anyway in case there was a pleasant surprise.

Would be interested in a looking glass solution so that I could run Windows 10 games under linux, but assume from your description above that i would be limited to using the iGPU.

output of ls-iommu.sh

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:3e30] (rev 0a)
IOMMU Group 10 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a305] (rev 10)
IOMMU Group 10 00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
IOMMU Group 10 00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
IOMMU Group 10 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
IOMMU Group 10 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-V [8086:15bc] (rev 10)
IOMMU Group 11 02:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 [144d:a808]
IOMMU Group 12 04:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 [144d:a808]
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 0a)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti Rev. A] [10de:1e07] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation TU102 High Definition Audio Controller [10de:10f7] (rev a1)
IOMMU Group 1 01:00.2 USB controller [0c03]: NVIDIA Corporation TU102 USB 3.1 Controller [10de:1ad6] (rev a1)
IOMMU Group 1 01:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU102 UCSI Controller [10de:1ad7] (rev a1)
IOMMU Group 2 00:02.0 Display controller [0380]: Intel Corporation Device [8086:3e98]
IOMMU Group 3 00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
IOMMU Group 4 00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
IOMMU Group 4 00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
IOMMU Group 4 00:14.3 Network controller [0280]: Intel Corporation Wireless-AC 9560 [Jefferson Peak] [8086:a370] (rev 10)
IOMMU Group 5 00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
IOMMU Group 6 00:17.0 RAID bus controller [0104]: Intel Corporation SATA Controller [RAID mode] [8086:2822] (rev 10)
IOMMU Group 7 00:1b.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #17 [8086:a340] (rev f0)
IOMMU Group 8 00:1c.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #1 [8086:a338] (rev f0)
IOMMU Group 9 00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port #9 [8086:a330] (rev f0)

1 Like

With a Samsung 970 Evo PCIE M.2 1 TB:

hdparm -tT /dev/nvme0n1

/dev/nvme0n1:
Timing cached reads: 42222 MB in 2.00 seconds = 21149.58 MB/sec
Timing buffered disk reads: 6636 MB in 3.00 seconds = 2211.95 MB/sec

This was originally going to be a SFF build in a DAN A4-SFX case… but I recently had to change out my 19" 12U Rack in my office, and I’m kinda wanting to throw everything in a 4U now, so I might be swapping out this motherboard/CPU. Ideally for pass-through I would have a 9900k or I’m hoping for some 12 core something from AMD Ryzen 3000 series.

Here’s the output of ls-iommu.sh on my ASUS ROG STRIX Z390-I GAMING motherboard w/ an i7-8700.

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation 8th Gen Core Processor Host Bridge/DRAM Registers [8086:3ec2] (rev 07)
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 07)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP102 [GeForce GTX 1080 Ti] [10de:1b06] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GP102 HDMI Audio Controller [10de:10ef] (rev a1)
IOMMU Group 2 00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
IOMMU Group 2 00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
IOMMU Group 2 00:14.3 Network controller [0280]: Intel Corporation Wireless-AC 9560 [Jefferson Peak] [8086:a370] (rev 10)
IOMMU Group 3 00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
IOMMU Group 4 00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake PCH SATA AHCI Controller [8086:a352] (rev 10)
IOMMU Group 5 00:1b.0 PCI bridge [0604]: Intel Corporation Device [8086:a340] (rev f0)
IOMMU Group 6 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:a338] (rev f0)
IOMMU Group 7 00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port 9 [8086:a330] (rev f0)
IOMMU Group 8 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a305] (rev 10)
IOMMU Group 8 00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
IOMMU Group 8 00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
IOMMU Group 8 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
IOMMU Group 8 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-V [8086:15bc] (rev 10)
IOMMU Group 9 04:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Device [15b7:5002]

Here is my setup of a recent build.

Specs:

MB: ASRock Taichi Ultimate z390
CPU: i9-9900k @ 5.0GHz on all cores
RAM: Trident Z Royal, 4266 kit of 2x8GB @ 4400-18-18-18-36
DISK1: Intel 600p 1TB, M2 NVME (spec @ 1800/560 read/write)
DISK2: WD Black, 500GB, M2 NVME (spec @ 3400/2200 read/write)
GPU1: RX470 8GB
GPU2: RX470 8GB

ls-iommu.sh:

xuw@homebox:~$ ./ls-iommu.sh
IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:3e30] (rev 0a)
IOMMU Group 10 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a305] (rev 10)
IOMMU Group 10 00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
IOMMU Group 10 00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
IOMMU Group 10 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
IOMMU Group 10 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-V [8086:15bc] (rev 10)
IOMMU Group 11 04:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Device [15b7:5002]
IOMMU Group 12 05:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
IOMMU Group 13 06:00.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 14 07:01.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 14 08:00.0 Network controller [0280]: Intel Corporation Wireless-AC 9260 [8086:2526] (rev 29)
IOMMU Group 15 07:03.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 15 09:00.0 Ethernet controller [0200]: Intel Corporation I211 Gigabit Network Connection [8086:1539] (rev 03)
IOMMU Group 16 07:05.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 17 07:07.0 PCI bridge [0604]: ASMedia Technology Inc. Device [1b21:1184]
IOMMU Group 17 0b:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02)
IOMMU Group 18 0c:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a5] (rev 03)
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 0a)
IOMMU Group 1 00:01.1 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x8) [8086:1905] (rev 0a)
IOMMU Group 1 00:01.2 PCI bridge [0604]: Intel Corporation Xeon E3-1200 v5/E3-1500 v5/6th Gen Core Processor PCIe Controller (x4) [8086:1909] (rev 0a)
IOMMU Group 1 02:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X] [1002:67df] (rev cf)
IOMMU Group 1 02:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [1002:aaf0]
IOMMU Group 1 03:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X] [1002:67df] (rev cf)
IOMMU Group 1 03:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 580] [1002:aaf0]
IOMMU Group 2 00:12.0 Signal processing controller [1180]: Intel Corporation Cannon Lake PCH Thermal Controller [8086:a379] (rev 10)
IOMMU Group 3 00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
IOMMU Group 3 00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
IOMMU Group 4 00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
IOMMU Group 5 00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake PCH SATA AHCI Controller [8086:a352] (rev 10)
IOMMU Group 6 00:1b.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port 21 [8086:a32c] (rev f0)
IOMMU Group 7 00:1c.0 PCI bridge [0604]: Intel Corporation Device [8086:a33c] (rev f0)
IOMMU Group 8 00:1c.6 PCI bridge [0604]: Intel Corporation Device [8086:a33e] (rev f0)
IOMMU Group 9 00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port 9 [8086:a330] (rev f0)


/dev/nvme0n1 (Intel 600p):
Timing cached reads: 39332 MB in 2.00 seconds = 19697.28 MB/sec
Timing buffered disk reads: 1738 MB in 3.00 seconds = 578.97 MB/sec

/dev/nvme1n1 (WD Black):
Timing cached reads: 39700 MB in 2.00 seconds = 19882.20 MB/sec
Timing buffered disk reads: 4148 MB in 3.00 seconds = 1382.46 MB/sec

1 Like

ls-iommu dump. First PCIe using video card. First and second m.2 slots are populated. Motherboard is a midly cheap ROG STRIX Z390-F. PCH PCIe Slot has 10gb nic. Verified 1.2GBs worth of speed earlier with a pci-e nvme adapter to test the PCH slot.

Doesn’t really answer the question of what would iommu groups look like if the cpu pcie lanes are split? Does it seperate iommu groups for a dual video card setup. Sorry. I use the igpu for host and pcie gpu for vms. I have noticed video lag in games during resource loading. But I’m still tweaing cpu pinning. And running this in Debian on experimental kernel could be the culprit too.

IOMMU Group 0 00:00.0 Host bridge [0600]: Intel Corporation Device [8086:3e30] (rev 0a)
IOMMU Group 10 00:1d.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port [8086:a330] (rev f0)
IOMMU Group 11 00:1f.0 ISA bridge [0601]: Intel Corporation Device [8086:a305] (rev 10)
IOMMU Group 11 00:1f.3 Audio device [0403]: Intel Corporation Cannon Lake PCH cAVS [8086:a348] (rev 10)
IOMMU Group 11 00:1f.4 SMBus [0c05]: Intel Corporation Cannon Lake PCH SMBus Controller [8086:a323] (rev 10)
IOMMU Group 11 00:1f.5 Serial bus controller [0c80]: Intel Corporation Cannon Lake PCH SPI Controller [8086:a324] (rev 10)
IOMMU Group 11 00:1f.6 Ethernet controller [0200]: Intel Corporation Ethernet Connection (7) I219-V [8086:15bc] (rev 10)
IOMMU Group 12 02:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
IOMMU Group 13 03:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 [144d:a808]
IOMMU Group 14 05:00.0 USB controller [0c03]: NEC Corporation uPD720200 USB 3.0 Host Controller [1033:0194] (rev 03)
IOMMU Group 15 06:00.0 Non-Volatile memory controller [0108]: Samsung Electronics Co Ltd NVMe SSD Controller SM981/PM981 [144d:a808]
IOMMU Group 1 00:01.0 PCI bridge [0604]: Intel Corporation Skylake PCIe Controller (x16) [8086:1901] (rev 0a)
IOMMU Group 1 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation GP104 [10de:1b82] (rev a1)
IOMMU Group 1 01:00.1 Audio device [0403]: NVIDIA Corporation GP104 High Definition Audio Controller [10de:10f0] (rev a1)
IOMMU Group 2 00:02.0 VGA compatible controller [0300]: Intel Corporation Device [8086:3e98]
IOMMU Group 3 00:14.0 USB controller [0c03]: Intel Corporation Cannon Lake PCH USB 3.1 xHCI Host Controller [8086:a36d] (rev 10)
IOMMU Group 3 00:14.2 RAM memory [0500]: Intel Corporation Cannon Lake PCH Shared SRAM [8086:a36f] (rev 10)
IOMMU Group 4 00:16.0 Communication controller [0780]: Intel Corporation Cannon Lake PCH HECI Controller [8086:a360] (rev 10)
IOMMU Group 5 00:17.0 SATA controller [0106]: Intel Corporation Cannon Lake PCH SATA AHCI Controller [8086:a352] (rev 10)
IOMMU Group 6 00:1b.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port [8086:a340] (rev f0)
IOMMU Group 7 00:1b.4 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port [8086:a32c] (rev f0)
IOMMU Group 8 00:1c.0 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port [8086:a338] (rev f0)
IOMMU Group 9 00:1c.7 PCI bridge [0604]: Intel Corporation Cannon Lake PCH PCI Express Root Port [8086:a33f] (rev f0)

hdparm -Tt output:

/dev/nvme0n1:
Timing cached reads: 41738 MB in 2.00 seconds = 20902.64 MB/sec
Timing buffered disk reads: 8632 MB in 3.00 seconds = 2877.09 MB/sec

/dev/nvme1n1:
Timing cached reads: 43446 MB in 2.00 seconds = 21759.81 MB/sec
Timing buffered disk reads: 8576 MB in 3.00 seconds = 2858.13 MB/sec

So this motherboard is a bit slow on the m.2 speeds prob sharing cpu pcie lanes with the m.2.

2 Likes

Errrm, WTF?

1 Like

I got an x99 system rather than Skylake way back when for virtualization reasons. I haven’t regretted it, but I wish Intel would stop this product segmentation madness. If they don’t, my next system will probably be AMD.

Hello and welcome from my horrible streaming dumpster fire earlier. :smiley:

1 Like

Just purchased a Gigabyte Z390 Designare and 9900K. Will see what iommu groups look like later today.

Based on what I’m seeing if your iGPU and primary graphics card on PCIE are in the same group. Can you passthrough the PCIE gpu to VM and still use iGpu on the host? Or would you need to run the ACS patch?

There is one z390 motherboard other than the GODLIKE and Aorus Xtreme that has the option to switch the PCI-e lanes from the chipset to the CPU in the BIOS. The Gigabyte z390 Designare. Though, its about the same price of the Master board. Its definitely a nice motherboard. I haven’t seen the iommu list on it yet.

Can you share the GB Z390 Designare iommu groups? Thanks!

Will get back to you by tomorrow @eurodomenii

2 Likes

OMG, a properly dirt cheap motherboard is actually usable (reviving this post for all other people moving their z370 paltform to lab duty)
With a Gigabyte Z370P-D3, i7-9700k, 2080ti and a single sata ssd my IOMMU looks like::

root@homelab:~# ./ls-iommu.sh 
IOMMU Group 0:
        00:00.0 Host bridge [0600]: Intel Corporation 8th/9th Gen Core 8-core Desktop Processor Host Bridge/DRAM Registers [Coffee Lake S] [8086:3e30] (rev 0a)
IOMMU Group 1:
        00:01.0 PCI bridge [0604]: Intel Corporation 6th-10th Gen Core Processor PCIe Controller (x16) [8086:1901] (rev 0a)
        01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU102 [GeForce RTX 2080 Ti Rev. A] [10de:1e07] (rev a1)
        01:00.1 Audio device [0403]: NVIDIA Corporation TU102 High Definition Audio Controller [10de:10f7] (rev a1)
        01:00.2 USB controller [0c03]: NVIDIA Corporation TU102 USB 3.1 Host Controller [10de:1ad6] (rev a1)
        01:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU102 USB Type-C UCSI Controller [10de:1ad7] (rev a1)
IOMMU Group 2:
        00:02.0 VGA compatible controller [0300]: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] [8086:3e98]
IOMMU Group 3:
        00:08.0 System peripheral [0880]: Intel Corporation Xeon E3-1200 v5/v6 / E3-1500 v5 / 6th/7th/8th Gen Core Processor Gaussian Mixture Model [8086:1911]
IOMMU Group 4:
        00:14.0 USB controller [0c03]: Intel Corporation 200 Series/Z370 Chipset Family USB 3.0 xHCI Controller [8086:a2af]
IOMMU Group 5:
        00:16.0 Communication controller [0780]: Intel Corporation 200 Series PCH CSME HECI #1 [8086:a2ba]
IOMMU Group 6:
        00:17.0 SATA controller [0106]: Intel Corporation 200 Series PCH SATA controller [AHCI mode] [8086:a282]
IOMMU Group 7:
        00:1b.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #17 [8086:a2e7] (rev f0)
IOMMU Group 8:
        00:1b.2 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #19 [8086:a2e9] (rev f0)
IOMMU Group 9:
        00:1b.3 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #20 [8086:a2ea] (rev f0)
IOMMU Group 10:
        00:1b.4 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #21 [8086:a2eb] (rev f0)
IOMMU Group 11:
        00:1c.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #1 [8086:a290] (rev f0)
IOMMU Group 12:
        00:1c.2 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #3 [8086:a292] (rev f0)
IOMMU Group 13:
        00:1c.3 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #4 [8086:a293] (rev f0)
IOMMU Group 14:
        00:1c.4 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #5 [8086:a294] (rev f0)
IOMMU Group 15:
        00:1d.0 PCI bridge [0604]: Intel Corporation 200 Series PCH PCI Express Root Port #9 [8086:a298] (rev f0)
IOMMU Group 16:
        00:1f.0 ISA bridge [0601]: Intel Corporation Z370 Chipset LPC/eSPI Controller [8086:a2c9]
        00:1f.2 Memory controller [0580]: Intel Corporation 200 Series/Z370 Chipset Family Power Management Controller [8086:a2a1]
        00:1f.3 Audio device [0403]: Intel Corporation 200 Series PCH HD Audio [8086:a2f0]
        00:1f.4 SMBus [0c05]: Intel Corporation 200 Series/Z370 Chipset Family SMBus Controller [8086:a2a3]
IOMMU Group 17:
        07:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 0c)
IOMMU Group 18:
        08:00.0 Network controller [0280]: Intel Corporation Wi-Fi 6 AX200 [8086:2723] (rev 1a)