[SOLVED] 7900 XTX Code 43 or How to get 7900 XTX to work on VFIO

Hello guys, I am trying to pass through a 7900 XTX but I keep getting code 43 in Winows.

I have Above 4G Decoding enabled and Resizable BAR Support disabled in the BIOS. I am on X399. Arch 6.1.12-arch1-1.

I have added the GPU VBIOS to the QEMU configuration so I don’t get the reset bug anymore. But the driver refuses to work.

These are my GRUB CMD options: quiet splash root=ZFS=srpool/enc/ROOT loglevel=3 zfs.zfs_arc_max=4294967296 rd.udev.log_priority=3 vt.global_cursor_default=0 amd_iommu=on iommu=pt rd.driver.pre=vfio_pci vfio-pci.ids=8086:10ed,1002:744c,1002:ab30,8086:105e video=efifb:off,vesafb:off,vesa:off amdgpu.ppfeaturemask=0xffffffff

What am I missing?

Edit: Fixed by adding the hyperv stuff from here: 6700xt Windows VM Drivers equal black screen - #2 by Janos

Edit2: This works for the first time you run the VM after every host boot. If I shut down the VM and then start it again, the display out works but I get constant GPU driver crashes when I try to game and I need to restart the host to make it work again. It seems like something doesn’t get reset and the driver crashes.

3 Likes

so you got the 7900XTX to work ?
now i am jealous since i had to return mine ans switched to a 4 :nauseated_face:8 :face_vomiting: because of the reset bug :frowning:

Maybe it would be nice to make a small write up of the steps so others can follow along getting the 7900 to work :smiley:

2 Likes

Got it working by:

  1. Disabling Resizable BAR in the BIOS. It would error out with Code 43 with ReBAR on, but that might be the limited ReBAR support on X399.

  2. Adding the hyperv section below in the VM XML:

  <features>
    <acpi/>
    <apic/>
    <hyperv mode="custom">
      <vendor_id state="on" value="1234567890ab"/>
    </hyperv>
    <vmport state="off"/>
    <ioapic driver="kvm"/>
  </features>
  1. Once booted to Windows, I used GPU-Z to dump the card’s VBIOS to a file. I did that while having the Code 43 error, it worked regardless. Then I copied the file over to the host and set the “rom file” section in the VM XML, at the GPU passthrough section:
    <hostdev mode="subsystem" type="pci" managed="yes">
      <driver name="vfio"/>
      <source>
        <address domain="0x0000" bus="0x44" slot="0x00" function="0x0"/>  *<- THIS IS WHERE MY 7900XTX IS PLUGGED INTO THE HOST*
      </source>
      <rom file="*FULL PATH TO YOUR VBIOS FILE*"/> *<- THIS IS THE PART THAT SOLVES THE RESET BUG*
      <address type="pci" domain="0x0000" bus="0x01" slot="0x00" function="0x0" multifunction="on"/>
    </hostdev>
1 Like

thank you so much for the write up!
so you did not need to edit the bios like with older nvidia cards?

May i suggest to change the title of the thread to sth like “How to get 7900 XTX to work on VFIO” for better SEO (:

That was close that I almost got 7900xtx but it was not in stock. Thus, I ended up 4070 Ti. It turned out a good thing that GPU passthrough works like a charm.

No problem! No need to edit the BIOS.

Added your suggestion to the title!

1 Like

A way to use the large BAR for the 7900XTX in the Windows guest VM:

Turn Resizable BAR OFF in BIOS.

Manually set BAR 0 to 32GB at the Linux host.
Manually set BAR 2 to 8MB at the Linux host. Anything larger than 8MB causes code 43 in Windows.

How do manually resize the BARs:

echo -n "0000:03:00.0" > /sys/bus/pci/drivers/vfio-pci/unbind  ## change 0000:03:00.0 to your GPU's address
echo 15 > /sys/bus/pci/devices/0000\:03\:00.0/resource0_resize ## numbers 1-15 correspond to different bar sizes. 15 is 32GB, for more check the reddit comment mentioned below
echo 3 > /sys/bus/pci/devices/0000\:03\:00.0/resource2_resize ## same as above
echo -n "0000:03:00.0" > /sys/bus/pci/drivers/vfio-pci/bind

How to automatically resize the BAR at each boot:
PCI passthrough via OVMF - ArchWiki See 10.24
Edit: The udev rule is not working for me ATM, working on a solution.

Reddit comment:

5 Likes

hello
i have a question. If you manage 7900XT? get working in virt WIN, then … does it suffer from reset bug? Are u able to reset/force reset VM anytime and get VM working again ? Is it working after WIN crash/BSOD ?

Thank you.

BR,
raven4

I tried a forced reset on the Windows guest VM and it caused the host to lock up and reboot.

Not sure if it is because of the 7900XTX or the USB controller I am passing through though.

wow really great find!

Hello! I have a question here. I think you didn’t used it as a single gpu passthrough? I have a 7900 xtx and somehow can’t get it to work as for now. As the start hook gets activated, I just get a blackscreen and with the end hook it doesnt re-activate for the linux host. I use a Rom, which I have exported while I was on Windows before switching to Linux.

You need to add a second virtual display and try to troubleshoot, see if there is a code 43 or something.