[SOLVED] Please help - Esxi Losing data-storage and VM's when enabling USB 3 pass-through

Software & Operating Systems
1

Hello,

I’m very new to esxi and trying to get my usb3 pass through to work. Every time I enable the onboard usb3 headers for passthrough and reset the system to take effect, I end up losing my data storage and my VM’s and have to reinstall esxi. I’ve been hitting my head on this for almost a week.

Here my setup:

ASUS ROG STRIX X399-E GAMING motherboard: https://www.amazon.com/gp/product/B0756VTD19/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

AMD Ryzen Threadripper 2950X Processor: https://www.amazon.com/AMD-Ryzen-Threadripper-Processor-YD295XA8AFWOF/dp/B07GFN6CVF/ref=sr_1_2?keywords=thredripper&qid=1560906808&s=gateway&sr=8-2

32GB’s CORSAIR Vengeance RGB PRO 16GB (2x8GB) DDR4 3600MHz: https://www.amazon.com/gp/product/B07D1XJ8J9/ref=ppx_yo_dt_b_search_asin_title?ie=UTF8&psc=1

ASUS ROG Strix GeForce RTX 2080: https://www.amazon.com/ASUS-Advanced-Overclocked-Graphics-ROG-STRIX-RTX-2080-A8G/dp/B07JPD4NZV/ref=sr_1_11?keywords=1080ti&qid=1560907063&s=gateway&sr=8-11 (Trying to use this as a gaming VM with USB 3 passthrough. this is in PciE slot 1)

Radeon HD 6450: https://www.amazon.com/Sapphire-Radeon-PCI-Express-Graphics-100322L/dp/B004WYV8Z6/ref=sr_1_14?keywords=ati+video+card&qid=1560907317&s=gateway&sr=8-14 (Use this for ESXI. This is in PciE slot 3 )

3x SSD’s setup in the following way:

1- 1TB SSD for VM’s

1- 512GB SSD for ISO’s (mount the ISO’s and install from there)

1- 256GB SSD for ESXI

For some reason I can’t get the onboard usb 3 ports to passthrough. I’ve tried to pass through all 3 onboard usb3 headers, then tried two of them, then with one and every time, I get the same results… I lose my data storage and my VM’s for some reason… yet esxi still boots?

Here the online manual for my board: https://images-eu.ssl-images-amazon.com/images/I/C1Jk738TByS.pdf of which on page 8 says I have 3 onboard USB headers… all of which esxi is picking up. Again, does not matter which order I try to pass through, I still lose access.

So, I did some research online and and people were saying NOT to do usb passthrough, but just to pass through the USB device. The issue here is that my mouse and keyboard are not showing up under the USB devices (of which I’m trying to pass through). This is a 2nd set mouse and keyboard with the esxi having its own.

I also bought this USB card: https://www.amazon.com/gp/product/B00FPIMICA/ref=ppx_yo_dt_b_asin_title_o00_s00?ie=UTF8&psc=1 and when trying to pass this through, when powering on the VM, I get an error saying the “Device 65:0.0 is not a passthrough device”

I then went out and bought a High Point 4-Port USB 3.0 (https://www.amazon.com/High-Point-PCI-Express-RocketU-1144D/dp/B015CQ8DCS ) and it’s not working either. Same error:

Power On VM Key

haTask-1-vim.VirtualMachine.powerOn-105

Description

Power On this virtual machine

Virtual machine (machine Name)

State Failed - Module ‘DevicePowerOn’ power on failed.

Errors

Module 'DevicePowerOn' power on failed.
Device 65:0.1 is not a passthrough device.
Failed to start the virtual machine.

Anyone have any ideas?

2

ESXi has issues passing through devices not directly connected through the CPU. The USB headers are through the chipset and not the CPU so it won’t pass them through.

For the USB cards, are you trying to passthrough everything in the IOMMU group? Or are you trying to only passthrough a couple of the USB ports?

Can you post screenshots of what the screen for passthrough devices shows? Believe it’s Manage > Hardware tab > PCI Devices

3

There is an option for “Enumerate all IOMMU NVRs” --If Memory serves-- That needs to be enabled before any kind of passthrough works right on ESXi… I’ve only tested one TR board (very similar, Asus Prime X399-A) with ESXi and I had to enable that before I could get passthrough to work at all. Also, I’d be interested if you can get SR-IOV working on ESXi with that variant of board, the prime x399-a I tested didnt work at all for SR-IOV on any platform. Asus was no help either.

4

@2bitmarksman Thank you so much for the reply! This is the 3rd website I asked on and the first reply I got.

For the USB cards, are you trying to passthrough everything in the IOMMU group? Or are you trying to only passthrough a couple of the USB ports?> Blockquote I’e tried both ways with no luck. I just tested with passing the High Point 4-Port USB 3.0 card through (all 4 heads) but only adding one to the VM and I still get the same error.

Capture
Capture.PNG1475×740 54.4 KB
Capture2
Capture2.PNG853×683 35.1 KB
Capture3
Capture3.PNG1670×457 30.5 KB

I do believe this is what you’re asking for?

@TheKurrgan I’m new to esxi and not really sure what you want me to test. If you can provide me some details, I’m happy to test it for you.

5

I need the entire list. Don’t filter the PCI Devices by ‘Passthrough capable’

6

I’d also like to 2nd what @TheKurrgan said. If there’s an option to enable IOMMU, make sure it’s set to enabled explicitly (if it’s on Auto, change to enabled)

7

Also, make sure these settings are set:

Fastboot - Disabled
CSM - Enabled
SecureBoot - Disabled
XHCI hand-off - Enabled
IOMMU - Enabled

8

@2bitmarksman @TheKurrgan

Thank you both so much! It booted! I did not have IOMMU enabled, upon enabling it, the system just worked! I owe you both a 12 pack. Please PM me your paypal account, or what ever and I’ll be happy to make a payment!

I can now move on to learning more about esxi and playing with it.

9

It’s good man, happy to help out. Also, ESXi can run headless (you don’t need the GPU once you’ve installed ESXi and can access it’s webgui)