I tried for quite some time now to add different GPUs to a macOS VM on an Ubuntu 21.04 host with Threadripper Pro and ASUS WRX80E board with qemu.
For Windows and Linux guests it works fine but for macOS I just couldn’t get it running on this system.
Was someone else able to virtualise macOS on a Threadripper Pro system? For the moment an issue with Threadripper Pro or this board is my last idea why it doesn’t work. With a WX5100 the macOS boot justs gets stuck and with the R9 270X the hosts restarts everytime during macOS boot. Without the card or in Linux and Windows everything works fine.
I am a bit out of ideas what else could be the issue here. Any hints where to debug or someone who might share their experience with virtualising macOS?
The problem with the 270X seems to be related to passing graphics and audio pci part via multifunction. Passing the separately work. So somehow macos does not like the virtual pci root bridge I think.
MacOS through more recent iterations, are very specific of [accepted] hardware involved [you know, CrApple practices]]. Even mimicking an Intel spec’d cutsheet [say for a pre-discontinued Mac Pro Tower], can still have its haggles… I wouldn’t want to know how more headache be encountered, trying to wrangle AMD components
It should be? I have a macOS VM setup on my X570/5950X. Using a W5500 but WX5100 is fine as well.
Do you have Above 4G Decoding or Resizable BAR on in the BIOS? I found turning those options broke my macOS VM when a GPU was passed through. No issues with Windows, but it broke macOS.
4G and Resizable BAR are activated in the BIOS. You needed to disable them to use your GPU in a macOS VM? Sadly my system does not boot without 4G support. Probably because of multiple GPUs with lots of RAM. Have to remove them before testing if disabling 4G makes the difference.
Yes, with those settings enabled it completely freezes up during macOS boot. I enabled verbose output in OpenCore and the part it halts on was random too. Everything worked fine after disabling 4G though.
Ok, this will need some time. But by the way. Did you manage to pass VGA and Audio on the same PCI bus with only function different? My R9 270X works if I pass VGA and Audio on different guest PCI busses. But when having it via multifunction on the same bus the host crashes. Windows and Linux guest work with multifunction. Not sure why macos crashes the host if they are on the same PCI bus as they should be.
Yes - I have multifunction on. This is my XML if it helps any - GPU is on bus 0x01
https://raw.githubusercontent.com/relm/vfio-config/master/etc/libvirt/macOS.xml
Thank you. If I might ask. What host hard and software are you running? Because I tried Skylake-Server but didn’t manage on my Threadripper Pro and ended up using “host” model which works up to now but I am not sure if it could lead to some problems with software/drivers that thinks to require an Intel cpu.
Ryzen 5950X and Ubuntu 21.04. In order to get Skylake-Server to work, I had to disable those functions in the XML listed under the <cpu> section.
You are using OpenCore correct? As I recall Clover had some issues using anything other than Penryn for CPU type. macOS can be picky about the number of cores passed through also. In my experience, 4 or 8 works, but if I try 6 it won’t. If you are passing through more than that I’m not sure what is considered valid.
Thanks for your input. For my Threadripper Pro I had to disable emrs and invpcid in addtion and now the Skylake-Server model works.
But PCI passthrough of my GPU on the same bus still crashes the host.
What is your 0x10 PCI device? Do you pass the host pci bridge or is it just some different card?
Yes I use the latest OpenCore.
0x10 PCI device is one of the USB controllers from my motherboard.
@x3sphere Thank you very much for your input. After disabling Re-Size BAR Support in the BIOS all the GPUs can be passed to the VM.
The R9 still can’t be passed on the same virtual PCIe bus but I don’t care at the moment as the new GPUs work fine now.
The WX 5100 works fine in both Linux and macOS now:)
The W 5500 seems to crash the host when switching from macOS guest to Ubuntu guest but not the other way around. It complains about can't change power state from D3hot to D0 (config space inaccessible). Have to look into this a bit more.
@bachandi Thank you for posting about splitting the video and audio for GPU passthrough. I have the same board (WRX80) as you and after two weeks of trying I could not pass AMD FirePro W7000 to macOS 12 Monterey. Passing video and audio together will make the host crash and restart every time like you described. Passing the video alone works. I have not been able to pass audio on different PCI bus so far.
Windows and Linux both work with multifunction just fine.
Would you have any details on how to pass the audio as well?
Or if you finally made the multifuction work for Macos as well?