[Solved] Linux preferring to use R9 7900 iGPU instead of 5600 XT dGPU (preferred)

So running furmark and monitoring both GPU usages using CoreCtrl and (perhaps more importantly) glx-gears --info - which explicitly says which GPU it’s rendering on, both are rendering using the iGPU.

EDIT: Solved: see my reply.

OS: Fedora 39
Hardware:

  • Mobo: MSI X670-P WiFi
    • BIOS: E7D67AMS.1A0 AKA 7D67v1A (not the most recent BIOS as that one is marked Beta, but the one before, released 2023-08-15.
  • CPU: AMD R9 7900X, 2nd monitor plugged in to DisplayPort to VGA adapter
  • Top x16 Slot (Running at x8 IIRC): RTX 2070, plugged into 2nd monitor on DisplayPort to VGA (which means I have to switch inputs on the 2nd monitor to view stuff on either iGPU or the 2070)
  • Middle x16 slot (IIRC x4 Gen4): 5600 XT; 1st monitor (primary in KDE) plugged in to DisplayPort to DVI adapter

BIOS settings have specified iGPU as primary over the other option PEG because PEG is for the top x16 slot, which I’m using for KVM hardware passthrough.

Troubleshooting steps so far:
Setting BIOS option Initiate Graphic Adapter to PEG causes Fedora 39 to fail to boot up properly; it gets to:
[ 22.xxxxxx] snd_hda_codec_ca0132 hdaudioC0D2: autoconfig for CA0132: line_outs=0 (0x0/0x0/0x0/0x0/0x0) type:line
(and a couple more info-lines about the audio config)
and then just hangs there. I can, however, press Ctrl+Alt+F2 and log in there; should I run the ls-iommu.sh script (obtained from an old forum post tutorialing VFIO, maybe the 2019 one?) I get a bunch of errors complaining about the PCIe device that is the 2070 (I think) and can’t seem to do issue any more commands. Ctrl+Alt+Delete successfully reboots Fedora.
Usually it also displays the following error before the audio error;
pcieport 0000:00:01.1: PME: Spurious native interrupt!,
which is usually just after:
Starting smb.service - Samba SMB Daemon...
sometimes it gets as far as
[ OK ] Created slice user-1000.slice - User Slice of UID 1000.
and sometimes also:
r8169 0000:06:00.0 invalid VPD tag 0x00 (size 0) at offset 0; assume missing optional EEPROM

Whereas setting the BIOS Initiate Graphic Adapter to IGD allows… presumably the X11 server to start.

I’m not entirely sure what the HybridGraphics toggle in the BIOS does, as enabling it doesn’t seem to affect whether the iGPU outputs a display when Initiate Graphic Adapter is set to either of PEG or IGD. It also doesn’t affect the “soft-hang” when it’s set to PEG. Maybe it’s because I’ve set the memory allocation to Force, 768M instead of one of the auto settings?

Current grub command line:
GRUB_CMDLINE_LINUX_DEFAULT="amd_iommu=on iommu=pt rd.driver.pre=vfio-pci vfio-pci.ids=10de:1f02,10de:10f9,10de:1ada,10de:1adb, radeon.cik_support=0 amdgpu.cik_support=1 video=efib:off"
Where the pci.ids starting with 10de are the 2070’s PCIe IDs
At one point I also had 1002:164e,1002:1640 and for some reason wasn’t being grabbed by vfio - now after some playing around with removing them and adding them back, and adding the most recent command to the end of grub, adding the iGPU back in causes the same “soft-hang” that booting with PEG rather than IGD causes.

Outputs of an older ls-iommu script from this forum (IOMMU groups seem fine):

./ls-iommu.sh | grep "VGA"

IOMMU Group 13 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106 [GeForce RTX 2070] [10de:1f02] (rev a1)
IOMMU Group 26 19:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev ca)
IOMMU Group 28 1a:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raphael [1002:164e] (rev c2)

./ls-iommu.sh | grep "1002"

IOMMU Group 24 17:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Upstream Port of PCI Express Switch [1002:1478] (rev ca)
IOMMU Group 25 18:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 XL Downstream Port of PCI Express Switch [1002:1479]
IOMMU Group 26 19:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev ca)
IOMMU Group 27 19:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 HDMI Audio [1002:ab38]
IOMMU Group 28 1a:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raphael [1002:164e] (rev c2)
IOMMU Group 29 1a:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Rembrandt Radeon High Definition Audio Controller [1002:1640]

./ls-iommu.sh | grep "10de"

IOMMU Group 13 01:00.0 VGA compatible controller [0300]: NVIDIA Corporation TU106 [GeForce RTX 2070] [10de:1f02] (rev a1)
IOMMU Group 13 01:00.1 Audio device [0403]: NVIDIA Corporation TU106 High Definition Audio Controller [10de:10f9] (rev a1)
IOMMU Group 13 01:00.2 USB controller [0c03]: NVIDIA Corporation TU106 USB 3.1 Host Controller [10de:1ada] (rev a1)
IOMMU Group 13 01:00.3 Serial bus controller [0c80]: NVIDIA Corporation TU106 USB Type-C UCSI Controller [10de:1adb] (rev a1)

Aha! Solved it! I was looking in the wrong place.

I also should’ve mentioned that I’m running KDE Plasma on X11.

According to this post here

I needed to edit my Xorg config, and specify the BusID in decimal, not hex (the output of lspci), for example 10-amdgpu.conf in /usr/share/X11/xorg.conf.d was

Section "OutputClass"
	Identifier "dGPU"
	MatchDriver "amdgpu"
	Driver "amdgpu"
	Option "HotplugDriver" "amdgpu"
EndSection

and now is

Section "OutputClass"
	Identifier "dGPU"
	MatchDriver "amdgpu"
	Driver "amdgpu"
	Option "HotplugDriver" "amdgpu"
EndSection

Section "Device"
    Identifier             "dGPU"
    Driver                 "amdgpu"
#   PCI 19:00.0 is in hex; it needs to be converted to decimal (25).
#   Apparently it can be kept as hex if skipping leading zero(s) and presumably hex characters are used
    BusID                  "PCI:25:0:0"
EndSection

When the output of lspci -nn -d 1002:731f is
19:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Navi 10 [Radeon RX 5600 OEM/5600 XT / 5700/5700 XT] [1002:731f] (rev ca)

1 Like