The PCIe Passthrough bug is back

Hi

Everytime I run this code:

#what’s up with the GPU
lspci -nkks 2d:00.0
#unbind GPU
echo “0000:2d:00.0” > /sys/bus/pci/devices/0000:2d:00.0/driver/unbind
#assign new ID to GPU
echo 148c 2391 > /sys/bus/pci/drivers/vfio-pci/new_id
#check if GPU is bound
lspci -nkks 2d:00.0

My system freezes up.
It worked before.
Only Java Script libraries were updated.
What could be the problem?
How can I stop loading the amdgpu driver at startup,
so the vfio driver can be assigned?

Troubleshooting Freezes:
added:
/etc/modprobe.d/local.conf:
alias pci:v00001002d000067DFsv0000148Csd00002391bc03sc00i00 vfio-pci
options vfio-pci ids=15b7:5011,1002:67df,1002:aaf0
No change
Usually a bios setting is responsible for the PCIe crash but both settings are disabled
-resize bar
-above 4GB
and I did not change anything in the BIOS beforehand.
It ran with stutters here and there but it ran.
Dialed down memory of from 32768 to 16384kiB
No dice.
Killed active kvm processes.
2275
1142
Yeeha!
Works again.
But now it doesn’t, again.
The PCI-STUB did not work either.

Any suggestions or mistakes in the code?

what motherboard?

try to enable iommu=on, pcie aer/acs on, ?

Hi Wendell

My Rig:

  • Mainboard: MSI X570A PRO.

  • CPU: Ryzen 5950X

  • RAM: 128GiB @ 3200MHz

  • host GPU: AMD Radeon HD5400

  • host NVME SSD: WD Blue SN550 2TB 4x PCIe 3.0 Chipset (Linux + Extra Storage)

  • guest GPU: AMD Radeon RX 590

  • guest NVME SSD: WD Blue SN850 1TB 4x PCIe 4.0 CPU (Win10 VM)

According to this code:

for d in $(find /sys/kernel/iommu_groups/ -type l sort -n -k5 -t/); do

n=${d#*/iommu_groups/*}; n=${n%%/*}

printf 'IOMMU Group %s ' "$n"

lspci -nns "${d##*/}"

done;

From here:

Shows that IOMMU is on.

dmesg | grep pci-stub
Shows nothing so the code is either wrong or something renders it ineffective.

If you really want to read up on my journey the main post is:

It is not a tutorial yet, but I included all errors I encountered.
Is rather convoluted but it tells how I got where I am now.

What’s the kernel cmd line also?

cat /proc/cmdline

BOOT_IMAGE=/boot/vmlinuz-5.10.0-23-amd64 root=UUID= :slight_smile:  ro amd_iommu=on iommu=pt kvm.ignore_msrs=1 vfio-pci.ids=15b7:5011,1002:67df,1002:aaf0

Still working on the mainboard rom dump.
flashrom -r file.rom
Threw me a curve ball:
flashrom --programmer=internal -r /home/user/2023.AUG.02_rom-file_MSI-X570A-PRO-mainboard.rom

flashrom v1.2 on Linux 5.10.0-23-amd64 (x86_64)
flashrom is free software, get the source code at https://flashrom.org

Using clock_gettime for delay loops (clk_id: 1, resolution: 1ns).
Found chipset "AMD FP4".
Enabling flash write... Disabling read write protection of flash addresses from 0xfc000000 to 0xfc05ffff failed.
Disabling read write protection of flash addresses from 0xfd000000 to 0xfd05ffff failed.
Disabling read write protection of flash addresses from 0xfe000000 to 0xfe05ffff failed.
Disabling read write protection of flash addresses from 0xff000000 to 0xff05ffff failed.
FCH device found but SMBus revision 0x61 does not match known values.
Please report this to [email protected] and include this log and
the output of lspci -nnvx, thanks!.
Could not determine chipset generation.PROBLEMS, continuing anyway
No EEPROM/flash device found.
Note: flashrom can never write if the flash chip isn't found automatically.

lspci -nnvx

00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Micro-Star International Co., Ltd. [MSI] X570-A PRO motherboard [1462:7c37]
        Flags: fast devsel
00: 22 10 80 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 62 14 37 7c
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Flags: fast devsel, IRQ 26
        Capabilities: [40] Secure device <?>
        Capabilities: [64] MSI: Enable+ Count=1/4 Maskable- 64bit+
        Capabilities: [74] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [c8] Vendor Specific Information: Len=18 <?>
00: 22 10 81 14 40 04 10 00 00 00 06 08 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 22 10 81 14
30: 00 00 00 00 40 00 00 00 00 00 00 00 ff 00 00 00

00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
        Flags: fast devsel, IOMMU group 0
00: 22 10 82 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 28, IOMMU group 1
        Bus: primary=00, secondary=01, subordinate=01, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: fcf00000-fcffffff [size=1M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Root Port (Slot+), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1453]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] L1 PM Substates
        Capabilities: [3c4] Designated Vendor-Specific: Vendor=1022 ID=0001 Rev=1 Len=44 <?>
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 83 14 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 01 01 00 f1 01 00 20
20: f0 fc f0 fc f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 12 00

00:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 29, IOMMU group 2
        Bus: primary=00, secondary=20, subordinate=2c, sec-latency=0
        I/O behind bridge: 0000c000-0000efff [size=12K]
        Memory behind bridge: fc400000-fcbfffff [size=8M]
        Prefetchable memory behind bridge: 00000000e0000000-00000000efffffff [size=256M]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Root Port (Slot+), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1453]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] L1 PM Substates
        Capabilities: [3c4] Designated Vendor-Specific: Vendor=1022 ID=0001 Rev=1 Len=44 <?>
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 83 14 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 20 2c 00 c1 e1 00 20
20: 40 fc b0 fc 01 e0 f1 ef 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 1a 00

00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
        Flags: fast devsel, IOMMU group 3
00: 22 10 82 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
        Flags: fast devsel, IOMMU group 4
00: 22 10 82 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:03.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 30, IOMMU group 5
        Bus: primary=00, secondary=2d, subordinate=2d, sec-latency=0
        I/O behind bridge: 0000f000-0000ffff [size=4K]
        Memory behind bridge: fce00000-fcefffff [size=1M]
        Prefetchable memory behind bridge: 00000000c0000000-00000000d01fffff [size=258M]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Root Port (Slot+), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1453]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] L1 PM Substates
        Capabilities: [3c4] Designated Vendor-Specific: Vendor=1022 ID=0001 Rev=1 Len=44 <?>
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 83 14 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 2d 2d 00 f1 f1 00 20
20: e0 fc e0 fc 01 c0 11 d0 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 12 00

00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
        Flags: fast devsel, IOMMU group 6
00: 22 10 82 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
        Flags: fast devsel, IOMMU group 7
00: 22 10 82 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
        Flags: fast devsel, IOMMU group 8
00: 22 10 82 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 32, IOMMU group 9
        Bus: primary=00, secondary=2e, subordinate=2e, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: [disabled]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Root Port (Slot-), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 84 14 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 2e 2e 00 f1 01 00 00
20: f0 ff 00 00 f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 01 12 00

00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
        Flags: fast devsel, IOMMU group 10
00: 22 10 82 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 33, IOMMU group 11
        Bus: primary=00, secondary=2f, subordinate=2f, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: fcc00000-fcdfffff [size=2M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Root Port (Slot-), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 84 14 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 00 2f 2f 00 f1 01 00 00
20: c0 fc d0 fc f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 01 12 00

00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
        Subsystem: Micro-Star International Co., Ltd. [MSI] X570-A PRO motherboard [1462:7c37]
        Flags: 66MHz, medium devsel, IOMMU group 12
        Kernel driver in use: piix4_smbus
        Kernel modules: i2c_piix4, sp5100_tco
00: 22 10 0b 79 03 04 20 02 61 00 05 0c 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 62 14 37 7c
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
        Subsystem: Micro-Star International Co., Ltd. [MSI] X570-A PRO motherboard [1462:7c37]
        Flags: bus master, 66MHz, medium devsel, latency 0, IOMMU group 12
00: 22 10 0e 79 0f 00 20 02 51 00 01 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 62 14 37 7c
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 0 [1022:1440]
        Flags: fast devsel, IOMMU group 13
00: 22 10 40 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 1 [1022:1441]
        Flags: fast devsel, IOMMU group 13
00: 22 10 41 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 2 [1022:1442]
        Flags: fast devsel, IOMMU group 13
00: 22 10 42 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 3 [1022:1443]
        Flags: fast devsel, IOMMU group 13
        Kernel driver in use: k10temp
        Kernel modules: k10temp
00: 22 10 43 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 4 [1022:1444]
        Flags: fast devsel, IOMMU group 13
00: 22 10 44 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 5 [1022:1445]
        Flags: fast devsel, IOMMU group 13
00: 22 10 45 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 6 [1022:1446]
        Flags: fast devsel, IOMMU group 13
00: 22 10 46 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Matisse Device 24: Function 7 [1022:1447]
        Flags: fast devsel, IOMMU group 13
00: 22 10 47 14 00 00 00 00 00 00 00 06 00 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

01:00.0 Non-Volatile memory controller [0108]: Sandisk Corp WD Black SN850 [15b7:5011] (rev 01) (prog-if 02 [NVM Express])
        Subsystem: Sandisk Corp WD Black SN850 [15b7:5011]
        Flags: bus master, fast devsel, latency 0, IRQ 43, NUMA node 0, IOMMU group 14
        Memory at fcf00000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [80] Power Management version 3
        Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+
        Capabilities: [b0] MSI-X: Enable+ Count=65 Masked-
        Capabilities: [c0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [1b8] Latency Tolerance Reporting
        Capabilities: [300] Secondary PCI Express
        Capabilities: [900] L1 PM Substates
        Capabilities: [910] Data Link Feature <?>
        Capabilities: [920] Lane Margining at the Receiver <?>
        Capabilities: [9c0] Physical Layer 16.0 GT/s <?>
        Kernel driver in use: nvme
        Kernel modules: nvme
00: b7 15 11 50 07 04 10 00 01 02 08 01 10 00 00 00
10: 04 00 f0 fc 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 b7 15 11 50
30: 00 00 00 00 80 00 00 00 00 00 00 00 0b 01 00 00

20:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse Switch Upstream [1022:57ad] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 24, IOMMU group 15
        Bus: primary=20, secondary=21, subordinate=2c, sec-latency=0
        I/O behind bridge: 0000c000-0000efff [size=12K]
        Memory behind bridge: fc400000-fcbfffff [size=8M]
        Prefetchable memory behind bridge: 00000000e0000000-00000000efffffff [size=256M]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Upstream Port, MSI 00
        Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [270] Secondary PCI Express
        Capabilities: [370] L1 PM Substates
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 ad 57 07 00 10 10 00 00 04 06 10 00 01 00
10: 00 00 00 00 00 00 00 00 20 21 2c 00 c1 e1 00 08
20: 40 fc b0 fc 01 e0 f1 ef 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 0a 01 1a 00

21:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 34, IOMMU group 16
        Bus: primary=21, secondary=22, subordinate=22, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: fcb00000-fcbfffff [size=1M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Downstream Port (Slot+), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:1453]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] L1 PM Substates
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 a3 57 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 21 22 22 00 f1 01 00 20
20: b0 fc b0 fc f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 12 00

21:01.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 35, IOMMU group 17
        Bus: primary=21, secondary=23, subordinate=23, sec-latency=0
        I/O behind bridge: 0000e000-0000efff [size=4K]
        Memory behind bridge: fca00000-fcafffff [size=1M]
        Prefetchable memory behind bridge: 00000000e0000000-00000000efffffff [size=256M]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Downstream Port (Slot+), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:1453]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] L1 PM Substates
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 a3 57 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 21 23 23 00 e1 e1 00 20
20: a0 fc a0 fc 01 e0 f1 ef 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 1a 00

21:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 36, IOMMU group 18
        Bus: primary=21, secondary=26, subordinate=26, sec-latency=0
        I/O behind bridge: 0000d000-0000dfff [size=4K]
        Memory behind bridge: fc900000-fc9fffff [size=1M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Downstream Port (Slot+), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:1453]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] L1 PM Substates
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 a3 57 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 21 26 26 00 d1 d1 00 00
20: 90 fc 90 fc f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 12 00

21:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a3] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 37, IOMMU group 19
        Bus: primary=21, secondary=27, subordinate=27, sec-latency=0
        I/O behind bridge: 0000c000-0000cfff [size=4K]
        Memory behind bridge: fc800000-fc8fffff [size=1M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Downstream Port (Slot+), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:1453]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] L1 PM Substates
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 a3 57 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 21 27 27 00 c1 c1 00 00
20: 80 fc 80 fc f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 ff 00 12 00

21:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 38, IOMMU group 20
        Bus: primary=21, secondary=2a, subordinate=2a, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: fc400000-fc5fffff [size=2M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Downstream Port (Slot-), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:1484]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [270] Secondary PCI Express
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 a4 57 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 21 2a 2a 00 f1 01 00 00
20: 40 fc 50 fc f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 0a 01 12 00

21:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 40, IOMMU group 21
        Bus: primary=21, secondary=2b, subordinate=2b, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: fc700000-fc7fffff [size=1M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Downstream Port (Slot-), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:1484]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [270] Secondary PCI Express
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 a4 57 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 21 2b 2b 00 f1 01 00 00
20: 70 fc 70 fc f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 05 01 12 00

21:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:57a4] (prog-if 00 [Normal decode])
        Flags: bus master, fast devsel, latency 0, IRQ 42, IOMMU group 22
        Bus: primary=21, secondary=2c, subordinate=2c, sec-latency=0
        I/O behind bridge: [disabled]
        Memory behind bridge: fc600000-fc6fffff [size=1M]
        Prefetchable memory behind bridge: [disabled]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Downstream Port (Slot-), MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [c0] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse PCIe GPP Bridge [1022:1484]
        Capabilities: [c8] HyperTransport: MSI Mapping Enable+ Fixed+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [270] Secondary PCI Express
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: pcieport
00: 22 10 a4 57 07 04 10 00 00 00 04 06 10 00 81 00
10: 00 00 00 00 00 00 00 00 21 2c 2c 00 f1 01 00 00
20: 60 fc 60 fc f1 ff 01 00 00 00 00 00 00 00 00 00
30: 00 00 00 00 50 00 00 00 00 00 00 00 0b 01 12 00

22:00.0 Non-Volatile memory controller [0108]: Sandisk Corp Device [15b7:501d] (rev 01) (prog-if 02 [NVM Express])
        Subsystem: Sandisk Corp Device [15b7:501d]
        Flags: bus master, fast devsel, latency 0, IRQ 24, NUMA node 0, IOMMU group 23
        Memory at fcb00000 (64-bit, non-prefetchable) [size=16K]
        Memory at fcb04000 (64-bit, non-prefetchable) [size=256]
        Capabilities: [80] Power Management version 3
        Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+
        Capabilities: [b0] MSI-X: Enable+ Count=17 Masked-
        Capabilities: [c0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [150] Device Serial Number 00-00-00-00-00-00-00-00
        Capabilities: [1b8] Latency Tolerance Reporting
        Capabilities: [300] Secondary PCI Express
        Capabilities: [900] L1 PM Substates
        Kernel driver in use: nvme
        Kernel modules: nvme
00: b7 15 1d 50 07 04 10 00 01 02 08 01 10 00 00 00
10: 04 00 b0 fc 00 00 00 00 00 00 00 00 00 00 00 00
20: 04 40 b0 fc 00 00 00 00 00 00 00 00 b7 15 1d 50
30: 00 00 00 00 80 00 00 00 00 00 00 00 0a 01 00 00

23:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Cedar [Radeon HD 5000/6000/7350/8350 Series] [1002:68f9] (prog-if 00 [VGA controller])
        Subsystem: PC Partner Limited / Sapphire Technology Cedar [Radeon HD 5000/6000/7350/8350 Series] [174b:e164]
        Flags: bus master, fast devsel, latency 0, IRQ 116, IOMMU group 24
        Memory at e0000000 (64-bit, prefetchable) [size=256M]
        Memory at fca20000 (64-bit, non-prefetchable) [size=128K]
        I/O ports at e000 [size=256]
        Expansion ROM at 000c0000 [disabled] [size=128K]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Legacy Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Kernel driver in use: radeon
        Kernel modules: radeon
00: 02 10 f9 68 07 04 10 00 00 00 00 03 10 00 80 00
10: 0c 00 00 e0 00 00 00 00 04 00 a2 fc 00 00 00 00
20: 01 e0 00 00 00 00 00 00 00 00 00 00 4b 17 64 e1
30: 00 00 a0 fc 50 00 00 00 00 00 00 00 0a 01 00 00

23:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Cedar HDMI Audio [Radeon HD 5400/6300/7300 Series] [1002:aa68]
        Subsystem: PC Partner Limited / Sapphire Technology Cedar HDMI Audio [Radeon HD 5400/6300/7300 Series] [174b:aa68]
        Flags: bus master, fast devsel, latency 0, IRQ 121, IOMMU group 24
        Memory at fca40000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Legacy Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
00: 02 10 68 aa 07 04 10 00 00 00 03 04 10 00 80 00
10: 04 00 a4 fc 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 4b 17 68 aa
30: 00 00 00 00 50 00 00 00 00 00 00 00 05 02 00 00

26:00.0 SATA controller [0106]: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:0612] (rev 02) (prog-if 01 [AHCI 1.0])
        Subsystem: ASMedia Technology Inc. ASM1062 Serial ATA Controller [1b21:1060]
        Flags: bus master, fast devsel, latency 0, IRQ 96, IOMMU group 25
        I/O ports at d050 [size=8]
        I/O ports at d040 [size=4]
        I/O ports at d030 [size=8]
        I/O ports at d020 [size=4]
        I/O ports at d000 [size=32]
        Memory at fc900000 (32-bit, non-prefetchable) [size=512]
        Capabilities: [50] MSI: Enable+ Count=1/1 Maskable- 64bit-
        Capabilities: [78] Power Management version 3
        Capabilities: [80] Express Legacy Endpoint, MSI 00
        Capabilities: [100] Virtual Channel
        Kernel driver in use: ahci
        Kernel modules: ahci
00: 21 1b 12 06 07 04 10 00 02 01 06 01 10 00 00 00
10: 51 d0 00 00 41 d0 00 00 31 d0 00 00 21 d0 00 00
20: 01 d0 00 00 00 00 90 fc 00 00 00 00 21 1b 60 10
30: 00 00 00 00 50 00 00 00 00 00 00 00 0a 01 00 00

27:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller [10ec:8168] (rev 15)
        Subsystem: Micro-Star International Co., Ltd. [MSI] X570-A PRO motherboard [1462:7c37]
        Flags: bus master, fast devsel, latency 0, IRQ 39, IOMMU group 26
        I/O ports at c000 [size=256]
        Memory at fc804000 (64-bit, non-prefetchable) [size=4K]
        Memory at fc800000 (64-bit, non-prefetchable) [size=16K]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [70] Express Endpoint, MSI 01
        Capabilities: [b0] MSI-X: Enable+ Count=4 Masked-
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Virtual Channel
        Capabilities: [160] Device Serial Number 01-00-00-00-68-4c-e0-00
        Capabilities: [170] Latency Tolerance Reporting
        Capabilities: [178] L1 PM Substates
        Kernel driver in use: r8169
        Kernel modules: r8169
00: ec 10 68 81 07 04 10 00 15 00 00 02 10 00 00 00
10: 01 c0 00 00 00 00 00 00 04 40 80 fc 00 00 00 00
20: 04 00 80 fc 00 00 00 00 00 00 00 00 62 14 37 7c
30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 00 00

2a:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Flags: fast devsel, IOMMU group 20
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [270] Secondary PCI Express
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
00: 22 10 85 14 00 00 10 00 00 00 00 13 10 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 22 10 85 14
30: 00 00 00 00 48 00 00 00 00 00 00 00 ff 00 00 00

2a:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c] (prog-if 30 [XHCI])
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:1486]
        Flags: bus master, fast devsel, latency 0, IRQ 45, IOMMU group 20
        Memory at fc500000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci
00: 22 10 9c 14 07 04 10 00 00 30 03 0c 10 00 80 00
10: 04 00 50 fc 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 22 10 86 14
30: 00 00 00 00 48 00 00 00 00 00 00 00 0a 01 00 00

2a:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c] (prog-if 30 [XHCI])
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:148c]
        Flags: bus master, fast devsel, latency 0, IRQ 41, IOMMU group 20
        Memory at fc400000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci
00: 22 10 9c 14 07 04 10 10 00 30 03 0c 10 00 80 00
10: 04 00 40 fc 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 22 10 8c 14
30: 00 00 00 00 48 00 00 00 00 00 00 00 0b 03 00 00

2b:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51) (prog-if 01 [AHCI 1.0])
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Flags: bus master, fast devsel, latency 0, IRQ 97, IOMMU group 21
        Memory at fc700000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/16 Maskable- 64bit+
        Capabilities: [d0] SATA HBA v1.0
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [270] Secondary PCI Express
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: ahci
        Kernel modules: ahci
00: 22 10 01 79 07 04 10 00 51 01 06 01 10 00 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 70 fc 00 00 00 00 22 10 01 79
30: 00 00 00 00 48 00 00 00 00 00 00 00 0a 01 00 00

2c:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51) (prog-if 01 [AHCI 1.0])
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Flags: bus master, fast devsel, latency 0, IRQ 98, IOMMU group 22
        Memory at fc600000 (32-bit, non-prefetchable) [size=2K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/16 Maskable- 64bit+
        Capabilities: [d0] SATA HBA v1.0
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [270] Secondary PCI Express
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
        Kernel driver in use: ahci
        Kernel modules: ahci
00: 22 10 01 79 07 04 10 00 51 01 06 01 10 00 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 60 fc 00 00 00 00 22 10 01 79
30: 00 00 00 00 48 00 00 00 00 00 00 00 0a 01 00 00

2d:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [1002:67df] (rev e1) (prog-if 00 [VGA controller])
        Subsystem: Tul Corporation / PowerColor Ellesmere [Radeon RX 470/480/570/570X/580/580X/590] [148c:2391]
        Flags: bus master, fast devsel, latency 0, IRQ 118, IOMMU group 27
        Memory at c0000000 (64-bit, prefetchable) [size=256M]
        Memory at d0000000 (64-bit, prefetchable) [size=2M]
        I/O ports at f000 [size=256]
        Memory at fce00000 (32-bit, non-prefetchable) [size=256K]
        Expansion ROM at fce40000 [disabled] [size=128K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Legacy Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [200] Physical Resizable BAR
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2b0] Address Translation Service (ATS)
        Capabilities: [2c0] Page Request Interface (PRI)
        Capabilities: [2d0] Process Address Space ID (PASID)
        Capabilities: [320] Latency Tolerance Reporting
        Capabilities: [328] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [370] L1 PM Substates
        Kernel driver in use: amdgpu
        Kernel modules: amdgpu
00: 02 10 df 67 07 04 10 00 e1 00 00 03 10 00 80 00
10: 0c 00 00 c0 00 00 00 00 0c 00 00 d0 00 00 00 00
20: 01 f0 00 00 00 00 e0 fc 00 00 00 00 8c 14 91 23
30: 00 00 e4 fc 48 00 00 00 00 00 00 00 0b 01 00 00

2d:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [1002:aaf0]
        Subsystem: Tul Corporation / PowerColor Ellesmere HDMI Audio [Radeon RX 470/480 / 570/580/590] [148c:aaf0]
        Flags: fast devsel, IRQ 10, IOMMU group 27
        Memory at fce60000 (64-bit, non-prefetchable) [disabled] [size=16K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [58] Express Legacy Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable- Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [328] Alternative Routing-ID Interpretation (ARI)
        Kernel driver in use: vfio-pci
        Kernel modules: snd_hda_intel
00: 02 10 f0 aa 00 00 10 00 00 00 03 04 10 00 80 00
10: 04 00 e6 fc 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 8c 14 f0 aa
30: 00 00 00 00 48 00 00 00 00 00 00 00 0a 02 00 00

2e:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Flags: fast devsel, IOMMU group 28
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
00: 22 10 8a 14 00 00 10 00 00 00 00 13 10 00 00 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 22 10 8a 14
30: 00 00 00 00 48 00 00 00 00 00 00 00 ff 00 00 00

2f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Flags: fast devsel, IOMMU group 29
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [270] Secondary PCI Express
        Capabilities: [2a0] Access Control Services
        Capabilities: [400] Data Link Feature <?>
        Capabilities: [410] Physical Layer 16.0 GT/s <?>
        Capabilities: [440] Lane Margining at the Receiver <?>
00: 22 10 85 14 00 00 10 00 00 00 00 13 10 00 80 00
10: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 22 10 85 14
30: 00 00 00 00 48 00 00 00 00 00 00 00 ff 00 00 00

2f:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c] (prog-if 30 [XHCI])
        Subsystem: Micro-Star International Co., Ltd. [MSI] X570-A PRO motherboard [1462:7c37]
        Flags: bus master, fast devsel, latency 0, IRQ 107, IOMMU group 30
        Memory at fcc00000 (64-bit, non-prefetchable) [size=1M]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable- Count=1/8 Maskable- 64bit+
        Capabilities: [c0] MSI-X: Enable+ Count=8 Masked-
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] Transaction Processing Hints
        Kernel driver in use: xhci_hcd
        Kernel modules: xhci_pci
00: 22 10 9c 14 07 04 10 10 00 30 03 0c 10 00 80 00
10: 04 00 c0 fc 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 62 14 37 7c
30: 00 00 00 00 48 00 00 00 00 00 00 00 0a 03 00 00

2f:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]
        Subsystem: Micro-Star International Co., Ltd. [MSI] X570-A PRO motherboard [1462:9c37]
        Flags: bus master, fast devsel, latency 0, IRQ 44, IOMMU group 31
        Memory at fcd00000 (32-bit, non-prefetchable) [size=32K]
        Capabilities: [48] Vendor Specific Information: Len=08 <?>
        Capabilities: [50] Power Management version 3
        Capabilities: [64] Express Endpoint, MSI 00
        Capabilities: [a0] MSI: Enable+ Count=1/1 Maskable- 64bit+
        Capabilities: [100] Vendor Specific Information: ID=0001 Rev=1 Len=010 <?>
        Capabilities: [150] Advanced Error Reporting
        Capabilities: [2a0] Access Control Services
        Capabilities: [370] Transaction Processing Hints
        Kernel driver in use: snd_hda_intel
        Kernel modules: snd_hda_intel
00: 22 10 87 14 06 04 10 00 00 00 03 04 10 00 80 00
10: 00 00 d0 fc 00 00 00 00 00 00 00 00 00 00 00 00
20: 00 00 00 00 00 00 00 00 00 00 00 00 62 14 37 9c
30: 00 00 00 00 48 00 00 00 00 00 00 00 0a 04 00 00

I will try to use a USB stick in the dedicated USB port to recover the mainboard firmware and reflash the system.

Maybe even install bookworm when my my RX 6600 arrives.
Could scrape enough euros together to upgrade.
Needed a better card for the VM anyway.

Maybe I can even revive my fallen brethren the FX9590 B3 :kneeling_sword_grave: (Kneeling in front of the grave of my brethren with my hands on the buckle of my sword and my head bowed to honor the fallen)

^^

Add video=efifb:off and vfio_pci.disable_vga=1 to your kernel command line. You might not need both so feel free to try them one at a time. Don’t forget to run update-grub after making any changes to /etc/default/grub (assuming you’re on Ubuntu/Debian).

And it wouldn’t hurt to add a softdep for amdgpu to your module config:

softdep amdgpu pre: vfio-pci

Don’t forget to run update-initramfs -k all -u after making any changes here.

1 Like

It’s working again.
update-initramfs -k all -u
did it.
I was wondering why the changes did not take effect from:
/etc/modprobe.d/local.conf:

alias pci:v00001002d000067DFsv0000148Csd00002391bc03sc00i00 vfio-pci
options vfio-pci ids=15b7:5011,1002:67df,1002:aaf0

Now I know.

Thanks for everybody who helped :slight_smile:

1 Like

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.