When I have the cable detached, it works ok, meaning, Nvidia is controlled by the vfio driver.
01:00.0 VGA compatible controller [0300]: NVIDIA Corporation AD103 [GeForce RTX 4080] [10de:2704] (rev a1)
Subsystem: Gigabyte Technology Co., Ltd Device [1458:40bc]
Kernel driver in use: vfio-pci
Kernel modules: nouveau, nvidia_drm, nvidia
01:00.1 Audio device [0403]: NVIDIA Corporation Device [10de:22bb] (rev a1)
Subsystem: Gigabyte Technology Co., Ltd Device [1458:40bc]
Kernel driver in use: vfio-pci
Kernel modules: snd_hda_intel
15:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Raphael [1002:164e] (rev c9)
Subsystem: Gigabyte Technology Co., Ltd Device [1458:d000]
Kernel driver in use: amdgpu
Kernel modules: amdgpu
When I am attaching the cable before boot, it is not, and then I have a blank screen. I can try to make it fail and give you the output then.