Trying to get pass-through working on x570 and ESX (or anything)

Software & Operating Systems VFIO
,
1

So I’ve upgraded my PC for the first time in about 8 years and I thought i’d be bold and have a go at running my primary desktop environment (windows) as a VM with Graphics, USB etc passed through. allowing me to run my primary desktop, my NAS, and any lab machines in one box.

My motherboard is a Asrock x570 Phantom gaming 4.

I’ve gone with a free vsphere 7 licence in the first instance.

So far I have figured out how to enable both SR-IOV. and IOMMU in the bios. (which was not easy, damn how many options do BIOS have nowadays)
And I’ive actually managed to pass through my NVME drive and install windows to it.

Under hardware > pci devices
I note that despite enabling SR-IOV in the BIOS everything says Not capable

The web gui is extremely buggy. If i try and select my GPU to pass through, it seems to try and auto select the 3 associated components of this device but they do an erratic dance where it rapidly selects and deselects indefinitely until i refresh the page. (I’ve tried 3 browsers)

Yesterday I managed to get to the stage where I had the GPU passed through.
I just needed to pass the USB3 hubs through so i could use my mouse and keyboard.
It was continually saying that pass through would be enabled on restart but when i restarted it still said that.
On the rare occasion I could get a USB hub passed through the VM would not start with the error:
“Failed to register the device pciPassthru1 for 10:0.3 due to unavailable hardware or software support.”

I tried updating to the latest bios to the latest revision which lists:
Update AMD AGESA Combo-AM4 V2 1.0.0.2

This did seem to result in the PCI IDs swapping round a bit and it broke my GPU pass through but no progress.

So im a bit stuck.

Is there anything further I can try?
I know these techs can require further manual config but I’m completely new to this so I don’t know where to start.

Would there be any benefit to trying ESX6.5, or perhaps proxmox?

Thanks.

2

ESXi 7.0 has barely been out of beta so ESXi 6.7 U3 might be a better choice.
Or you could try Xen or KVM.

3

Well… I’m now running on 6.7, and I’ve managed to pass the GPU through successfully, and get it working with:
hypervisor.cpuid.v0 = “FALSE”
pciHole.start = “2048”

The only big issue I have now is how to pass through the USB.
I stil get:
Failed to register the device pciPassthru1 for 10:0.3 due to unavailable hardware or software support

4

I can’t help you much with ESXI. Generally ESXI loves to work with certified devices and there will be very few people trying to passthrough on unsupported platform.
I would recommend you to try with KVM. For example ProXmox. Passthrough is much more common with KVM and you will have much resource to use and people to help.
One more general advice - I have used ProxMox in the configuration you are planning to use - VM for desktop + servers on the same host. It works, it is possible but I would not recommend it in general. You have to pick your hardware carefully to avoid restarting the host if you have to restart the workstation VM. Also you have to take good care to isolate the workstation VM in order to avoid latency spikes during gaming or watching youtube videos for example.

5

Update. Ive got a trial of unraid running at the moment and that seems to be running well as I wanted it.