Vega passthrough on unRAID. Completely lost

[pixelperfect]

So, I’ve been poking at this system for a good couple days now and I’m really really lost.

I have passthrough working with my 1060 3GB on the same VM. That isn’t the issue. I understand passthrough at a base level. Stub the card, provide the ROM because it’s Slot 1, and party.

My system:

Motherboard: MSI B350M Gaming Pro
CPU: Ryzen 7 1700X
RAM: 32GB (2x16GB) Kingston RAM
GPU: MSI AirBoost OC Vega 64 8GB
Host OS: unRAID 6.6.7-stable and 6.7.0-rc7

Mind, this config works fine with a GTX 1060 3GB from MSI. Works great, actually. I just wanted more power for VR stuff and had this Vega 64 as an eGPU for my Macbook Pro and wanted to try it out as the card in my VR system before bothering to set it up with Bootcamp and dealing with issues related to eGPU latency and VR, as the 1060 was laggy with my Rift.

If I’m being honest, I’m almost tempted to switch from unRAID to WinServer2016, but I rely heavily on Docker, and all my disks are btrfs and I don’t have space to migrate all that data if I wanted to.

I’m running kernel version 4.19.33, QEMU 3.1, and the latest packages provided by unRAID

Best I can get is error 31 in Windows, which is to say, not very far. Trying to install the Radeon driver locks the system. Same error under ESXi, actually.

The ROM file doesn’t seem to make much of a difference.

[TheCakeIsNaOH]

Recent AMD cards have a reset bug. Vega is definitely affected. It is a hardware issue, so it would require a major firmware rework at a minimum on AMD’s part to fix, and they have not fixed even with the Radeon VII.

If you reboot/reset/turn off+ turn on the VM, you need to reboot the host before it will work. The issue is that the card does not reset properly without a full bare-metal system restart, which is fine normally, but is problematic in a VM.

This thread has some info-

Looking Glass - Changelog

I have managed to get some traction on the state of VEGA reset. So far the word is that nothing has been done to address this. However it seems there might be a way to fix the reset by means of a pci quirk for these particular cards, if so I will implement this ASAP and try to get the patch upstreamed.

[pixelperfect]

See, I’ve heard a lot about that, but I’ve also seen a bunch of threads on people getting KVM to work for at least the first boot.

This thread discussed an issue with the PCI device itself Threadripper 2990wx + Vega 64 + KVM GPU passthrough freeze on Apple Logo

This thread on the unRAID forums has the card booting https://forums.unraid.net/topic/79420-vm-windows-10-little-dropouts/

And in these forums Threadripper GPU Passthrough working with Vega

And I can’t find the post now, but I’ve definitely read that 4.19 helped with the reset bug, not that i even care. I just wanted to get the VM booting once and not dying the moment the driver tried to install

[TheCakeIsNaOH]

I do not know what is wrong if it still does not work the first vm boot after a host reboot.

[mo-mobile]

I was facing similar issues with a vega 56 on B450 Board.
The one setting hat seems to work is:
-Disable legacy boot options (for reasons I fail to understand)
-Save the ROM of my GPU in GPU-Z on baremetall install
-rename the rom to Vega56.rom in my case (this was necessary and did not work when named “Vega 10.rom”
-passthrough rom
-Stub GPU and HDMI Sound (for me they need to be in different IOMMU groups since HDMI sound can’t be passed through successfully)
-Not pass through HDMI sound
-Run Q35-3.1 (i440fx didn’t work)

Well…at this point I really thing the hole discussion is a hugh mess since Kernel changes, Bios changes and a lot of confusion in the process generated a situation where common configurations are a rare that every case seems to be different.

But I do hope that I might notch you or someone in the right direction.

[pixelperfect]

That’s really interesting that not passing audio worked for you. I’m guessing resetting your VM still doesn’t work without resetting the host?