Thunderbolt 3 with Threadripper Pro on a Lenovo Thinkstation?

Can anyone help me get a GC Titan Ridge 2.0 card working on a Lenovo Thinkstation P620 with a Threadripper Pro? My main use for Thunderbolt would be to connect to external high speed SSD drives – I am not so interested in running a display off it.

I am running Windows 10 (Version 10.0.19042 Build 19042) on a new Lenovo Thinkstation P620, with a Threadripper Pro 3955WX processor, 32 GB RAM, and an AMD Radeon Pro WD5700 graphics card. BIOS etc is up to date.

I have added a Gigabyte GC-Titan Ridge 2.0 card for Thunderbolt 3.0 connectivity, connecting the 5-pin header cable to the Thunderbolt header on the motherboard, and the USB 2.0 cable to the USB pins on the motherboard; and connected the power cables.

Although the motherboard has a Thunderbolt header, there is no option in the BIOS for Thunderbolt settings.

I installed the latest drivers from the Gigabyte website. Thunderbolt Control Center is installed, with version levels as below:

screenshot1

I am having a problem getting Windows to recognise my Samsung X5 disk when it is attached to a Thunderbolt port on the card. (The Samsung X5 works fine when attached to my HP Zbook laptop, which has a built-in Thunderbolt port. So it’s not a problem with the drive or the cable being faulty.)

On the Lenovo P620, Windows Device Manager can see the Thunderbolt™ Controller - 15EB under System Devices, and the Thunderbolt Control Center can detect the Samsung X5 when it is plugged into the port on the card, as this screenshot from Thunderbolt Control Center shows:

screenshot2

But although Thunderbolt Control Center can see the Samsung device, Windows cannot seem to see it, or recognise it as a drive. It does not appear under drives in Device Manager; it does not show up in Windows Explorer, and the Disk Management tool doesn’t see it either. Nor is it visible using the List Disk command from the command-line Disk Part tool.

Any thoughts on how I might get Windows actually to use the Samsung X5 as a disk?
Any help would be greatly appreciated!

1 Like

with that security level you have to “authorize” the device. try right clicking the tray icon??

I already “approved” it in the Thunderbolt Control Center (sorry, should have said that) - was that what you meant? Or is there some different kind of authorisation needed?

yeah, once you “approve” it in the tb center, it should show up in device manager. Any ? in device manager?

If you had the bios option you could turn off the fancy thunderbolt security mode and that’d probably fix it. Does the software give you the security option choices? if so try to select it there. It has a higher chance of working on the low security mode.

1 Like

That’s the problem - even though visible to Thunderbolt Control Center, and approved in it, the drive does not show up in Device Manager. There’s power to the drive (its light comes on, and it gets warm).

There doesn’t seem to be an option in Tb Control Center to set a low security mode - just “Always Connect” or “Connect Only Once” - I’ve tried both, to no avail.

If I boot up the computer with the Samsung X5 plugged into the Thunderbolt card, it doesn’t appear in Tb Control Center; but as soon as I take it out and plug it in again with the machine running, it does show up in Tb Control Center. But it still doesn’t show up as a disk in Device Manager.

I’m wondering if the lack of Thunderbolt configuration in the BIOS means that although there is a a Thunderbolt header on the motherboard, I’d need to treat the setup as though there wasn’t, and jumper pins 3 to 5 on the cable. I found a post from FurryJackman elsewhere on this board at

that said

"Yes, it is necessary to jumper the card on a unsupported motherboard without the header.

"However, on motherboards with the header, it only works in SPECIFIC slots because the PCH has to communicate with the controller to have it power on when a device is connected. With no devices connected, the default is to keep it powered down. Without BIOS configuration on unsupported boards, this means it’s impossible to wake the controller.

"To change settings on the controller itself, you need a fully supported board to change settings and ASUS Prime Intel boards are pretty much the go to for that.

“If you lack config settings in your UEFI, you may as well jumper your Titan Ridge controller rather than use the official header. The official header is to write configuration settings to the controller and to power on the controller on demand. If your UEFI can’t configure the controller, jumpering after configuring on an official board is better than using the official header.”

But then I have also seen stuff about needing to flash back to NVM 23 - my setup is using NVM50. And I have no idea whether the drivers from the Gigabyte website are DCH drivers that people tell you to avoid, or not…

So before I start trying to jumper cables, or flash the firmware back to NVM23, I thought I should seek the advice of people who know what they are doing, since I don’t here!

Update 19 Feb 2021:

  1. So, I unplugged the Titan Ridge header cable from the Thunderbolt header on the motherboard, and jumpered the pins 3 to 5 on the header cable. I got exactly the same result as when the cable was plugged into the Thunderbolt header: Thunderbolt Control Center can see the card; Device Manager can see the Thunderbolt controller but can’t see the Samsung X5 as a disk when it is plugged in; nor can Windows.

  2. I then uninstalled the DCH versions of the Thunderbolt drivers and flashed back to NVM23 using the utility posted at https://egpu.io/forums/postid/87833/

  3. I downloaded and installed the Intel legacy drivers (non-DCH) from
    Download Thunderbolt™ 3 Legacy Driver for Windows® 10 for Intel® NUC

Device Manager can see the Thunderbolt controller:

and the Intel Thunderbolt Software can see the controller too, with both its ports, but (unlike Thunderbolt Control Center with the DCH drivers) it can’t see the device attached to the controller:

Unfortunately the Thunderbolt software doesn’t seem to have any options for approving the device since it can’t see it. It doesn’t let me do anything by either left- or right-clicking on Port 1 or Port 2, and doesn’t have any menu options to do anything…

So it appears that in both setups - (1) NVM50, DCH drivers with Thunderbolt Control Centre, or (2) NVM 23, non-DCH drivers, Thunderbolt Software - Device Manager can see the Thunderbolt controller but not any disks attached to it. In setup (1) Thunderbolt Control Centre can see the Samsung X5 drive when it is hotplugged in, and allows me to approve it; but Windows still can’t see it. In setup (2) the Thunderbolt software can’t see the drive when it is attached.

Any thoughts, anyone? Many thanks in advance…

2 Likes

Now I have no idea if this will help, but fellow IT Pro here, when i was working at a university a year ago we supported a herd or so of Dell XPS laptops with Thunderbolt, and we CONSTANTLY had issues with getting Thunderbolt “working right” on them. Issues with docks, devices, firmware on docks, etc… We had trouble determining a tried and true method but what always generally seemed to do it for us was updating the bios, reimaging, then slowly applying and updating drivers for the system before connecting the device. Wish you the best with this, saw the title and was hoping for good news b/c TR Pro is def a very exciting product for the enterprise IMO.

Frontinus, you are boldly going where no one has gone before. I have been wondering if what you are trying could work.

My guess is that a Lenovo P620 BIOS update is needed to properly enable Thunderbolt and that won’t happen until they have their own add-in card ready for sale. When the P620 was announced last July, Lenovo stated that a Thunderbolt option was scheduled for “Early 2021”. Hopefully that uses the new TB4/USB4 chipset. But having the option to use the Gigabyte card would be nice.

Success !!! Of a sort…

So, I really don’t quite know how I did it, which is going to make it a little difficult for others to replicate. Having done the various steps in my last post (jumpered the cable; installed the legacy non-DCH drivers), I realised that the Thunderbolt Software was still showing a firmware version of NVM 50, not the 23 I thought I had flashed back to.

So since I was still running on 50, I thought I’d go back to Thunderbolt Control Centre where I could at least see the device, even if not access it. So I uninstalled the legacy drivers, using Wendell’s uninstall tricks from his original post:

and then used the
pnputil /enum-drivers > c:\drivers.txt

to see what tbt-related drivers were still there, and then
pnputil /delete-driver oemXX.inf /force /uninstall

(where XX is a number) to delete all the oem.inf drivers
I think I still needed to uninstall the Intel Thunderbolt Software using Control Panel.

After that, I rebooted, and reinstalled the latest Gigabyte drivers (these are the DCH ones), and downloaded Thunderbolt Control Center from Microsoft store.

That got me back to the state I had been in before, where I could see the device in Thunderbolt Control Center, running NVM50, but still not actually do anything useful with it, except approve it or unapprove it, pointlessly .
Next step was to use the flash utility to flash back the firmware to NVM23. I ran this and it said it failed. I ran it another time and it seemed to crash out of the program. But when I next launched Thunderbolt Control Center, it had in fact reverted to NVM 23:

I could see the Samsung drive attached, as before:

and approve it, as before:

But I still couldn’t see it in Windows.

By now the attentive reader (assuming anyone even gets this far without falling asleep) will have realised that there is one permutation I had not yet tried - the legacy drivers and Intel Thunderbolt Software (not Control Center) with NVM 23.

So, I uninstalled the DCH drivers using the method above (Wendell + pnputil), uninstalled Thunderbolt Control Center, and rebooted the machine.

I ran pnputil /enum-drivers > c:\drivers.txt again and was just going to open the drivers.txt file in Windows Explorer to check if the Thunderbolt drivers had indeed been removed, when I saw that Windows Explorer was now showing a new drive, Samsung X5 (E:)
Hallelujah!

But here’s the weird bit:

Device Manager now shows no Thunderbolt Controller installed under System devices where it used to be. The Gigabyte card evidently shows up as a problematic Base System Device under Other Devices, saying its drivers are not installed:

Enumerating the installed drivers to drivers.txt shows that the following 3 drivers ARE in fact installed - Windows must have reverted to them as I forgot to scrub the backup package from System32\Drivers\Filestore

Published Name: oem10.inf
Original Name: tbtp2pndisdrv.inf
Provider Name: Intel® Corporation
Class Name: Network adapters
Class GUID: {4d36e972-e325-11ce-bfc1-08002be10318}
Driver Version: 04/12/2020 1.41.890.0
Signer Name: Microsoft Windows Hardware Compatibility Publisher

Published Name: oem11.inf
Original Name: tbthostcontrollerhsacomponent.inf
Provider Name: Intel® Corporation
Class Name: Software components
Class GUID: {5c4c3332-344d-483c-8739-259e934c9cc8}
Driver Version: 04/12/2020 1.41.890.0
Signer Name: Microsoft Windows Hardware Compatibility Publisher

Published Name: oem12.inf
Original Name: tbthostcontrollerextension.inf
Provider Name: Intel® Corporation
Class Name: Extensions
Class GUID: {e2f84ce7-8efa-411c-aa69-97454ca4cb57}
Extension ID: {b4daa161-39a4-49ea-bc66-7b05a9ddbb16}
Driver Version: 04/12/2020 1.41.890.0
Signer Name: Microsoft Windows Hardware Compatibility Publisher

Task manager tells me that Thunderbolt Service is running, which I suppose is what is providing the service that allows Windows to see the disk. So, Device Manager can’t see a thing it recognises as a Thunderbolt controller; I can’t actually open Thunderbolt control software to approve devices or see what is connected - BUT Windows can see the disk and that is basically what I need. So as I write this I am copying 1.5 Tb (terabytes, not thunderbolt!) of data to the Samsung X5 and I seem to have got working what I need to get working - even if I don’t really know how!

I haven’t tried using a display via thunderbolt, or charging from it - just data transfer. But that now shows that it can be done on a Lenovo Thinkstation P620 with a Gigabyte GC-Titan Ridge 2.0 !

In summary, I think my current setup is now:
Gigabyte GC-Titan Ridge 2.0 with jumpered cable
NVM23
Intel DCH drivers version 1.41.890.0
NO Thunderbolt Control Center installed

Thanks to Wendell, poomer and Robert for their suggestions and encouragement !

4 Likes

Awesome! Thanks for sharing what you are doing.

Robert, yes I am sure what it really needs is the BIOS update. I (and others) have been asking on the Lenovo Community Thinkstation forum about this and the very helpful Lenovo people say that Lenovo is working on a solution but can’t give a release date.
Since there is a Lenovo PCIe 3.0 Thunderbolt add-in card that works with other Thinkstation models (P520, P720, P920), and there’s a Thunderbolt header on the motherboard, you’d think the hardware wasn’t the issue, so like you I’d guessed it would need a BIOS update, and I couldn’t be bothered to wait for that.
Of course, if they do bring out a certified solution based on Thunderbolt 4 over PCIe 4.0, that really would be worth it!
But for now it seems that it really is possible to use a 3rd-party solution, if you don’t mind spending a couple of days playing with different combinations of drivers, flashing firmware, and possibly even jumpering pins.
(It’s possible that the pin jumpering wasn’t necessary, and it’s the NVM23 firmware with DCH drivers that did it, but by this stage I just don’t know…)

2 Likes

I’m going to have a drink now to celebrate. :grinning:

2 Likes

@poomer thank you very much for doing me the honour of assuming I’m a fellow IT Pro. I was once, about 30 years ago, but for the last 17 years I’ve been a professor of archaeology, though still dabbling with computers perhaps more than I ought…

2 Likes

Yes this older “less secure” approach is how I did it all the way back on threadripper 2000 series. Some of what we see here may be an attempt to put the genie back in the bottle. But also Intel’s driver team has somehow made things much worse with dch drivers than the older non dch versions…

2 Likes

Well, that turned out not quite to be the end of the story. After using the device for a day (and backing up from it!), with some trepidation I tried a power down and cold boot. As I had feared, the setup didn’t survive it - when the machine came back on, it no longer saw the Samsung X5 drive in Windows.

Following various bits of systematic experimentation this morning, this appears to be the solution to get it to survive a warm boot, though I still can’t get it to persist across a cold boot:

Just to recap, so that people trying this for the first time can start from here, this assumes that the GC-Titan Ridge 2.0 card is installed in a Lenovo P620 Thinkstation (in my machine, it’s in slot 6, as the power cables won’t reach the card across the GPU in slot 3 if it’s in another slot). The USB 2.0 cable is connected to the USB 2.0 header on the motherboard (ignoring the leftmost 2 pins since Lenovo uses an 11-pin USB 2.0 header; the extra pins are apparently “a combination of ground and GPIO that is used for cable detection/verification” - see Thinkstation S30 issue with 11 pin USB header on Motherboard (BT)-English Community )
The Thunderbolt header cable is not plugged into the Thunderbolt header on the motherboard, since there is no BIOS option to control it. Instead, pins 3 to 5 on the header cable are jumpered with a paperclip (the exposed part of which is wrapped around in Sellotape to stop it shorting anything else).
So, with the card physically installed, it should show up in Device Manager as a Base System unit with problems:

This appears to be the sequence to get it working:

  1. Install the Intel DCH 1.41.890.0 drivers via Device Manager from the Base System unit that has problems. These are the drivers from the Gigabyte website, not the ones that come on the CD with the card. (Don’t install the ones on the CD.)
    Flash the firmware back to NVM23 using the utility posted at https://egpu.io/forums/postid/87833/

  2. Reboot.
    [The Thunderbolt controller should now show up in Device Manager as installed properly. Device Manager also shows an Unknown USB Device as having a problem (Device Descriptor Request Failed). I guess this is the internal USB 2.0 connection?]

  3. Install Thunderbolt Control Center from Microsoft Store.
    https://www.microsoft.com/en-us/p/thunderbolt-control-center/9n6f0jv38ph1?activetab=pivot:overviewtab

  4. Launch Thunderbolt Control Center; it sees the Thunderbolt controller, and a device (in my case, a Samsung X5 drive) when hotplugged.
    [Device Manager does not now show a problem with an Unknown USB device. It has just disappeared from the list. Therefore Thunderbolt Control Center must have installed the relevant USB driver to fix it.]

  5. Now we need to remove Thunderbolt Control Center, using the PowerShell in Administrator mode:

image

  1. Reboot.
    Success! Now we see the Samsung X5 as drive E:

This will persist across a warm reboot, but not across a power down and cold boot.
If you need to do a cold boot, then repeat steps 3-6 above.

2 Likes

I guess the explanation for this is to do with the kind of security settings @Wendell mentioned in his various posts above.
It seems that Thunderbolt Control Center needs to be installed to deal with the USB 2.0 connection, it appears, but it then needs to be removed in order to avoid authorisation problems.
I am guessing here - but if Thunderbolt Control Center reads the card via the PCIe slot, but sends approval information for a device to the card over the Thunderbolt header cable, then it can’t do it in this setup, because either the header cable is connected but not configured in BIOS, as there is no BIOS option for Thunderbolt yet in a P620; or it is not connected and the pins are jumpered (as in my setup). So either way, Thunderbolt Control Center can see the card via the PCIe slot, but can’t action any authorisation of devices, even though it presents the user with the software options to authorise them.
So the Thunderbolt Control Center has to be uninstalled to bypass this authorisation problem. And fortunately the uninstallation via get-appackage [asterisk]thunder[asterisk] |remove-apppackage seems to leave behind whatever Thunderbolt Control Center did to make the USB 2.0 header work, which is presumably crucial for Windows to see the drive?

Now, any suggestions on what to do to make the successful setup persist across a power-down and cold boot? I suspect it may need things doing in the Windows Registry, but I am way out of my depth here…

This Powershell script is my best effort to automate the process of restoring Thunderbolt services after a cold boot. It launches the Microsoft Store via a web browser (change the path and program name to the web browser you are using), allowing you to download and install Thunderbolt Control Center.
You must launch Thunderbolt Control Center, and unplug and replug the drive or device so Thunderbolt Control Center recognises it. Then go back to the Powershell window and hit Enter to uninstall Thunderbolt Control Center and reboot.
When the computer restarts you should be able to see your Thunderbolt device in Windows.

Here is the script, uploaded as a txt file because I can’t upload a .ps1 file; you’ll want to save it with a .ps1 extension:

InstallAndThenRemoveThunderboltControlCenter.txt (3.1 KB)

1 Like

Curious, Has this brute-force work around still been good for you? I’m in desperate need to get thunderbolt working on my P620 for an upcoming video editing project I have, using an external 64TB raid drive, that’s Thunderbolt 3, and not USB-C.

-BG

I’ve been using the GC-TITAN RIDGE AIC with my ThinkStation P620 for some time without the issues you are experiencing. I enabled Thunderbolt support and disabled the security the in the BIOS. These settings are hidden in the HII BIOS menu. However, you have several different tools at your disposal to change the BIOS settings. The “Windows BIOS setting tool” is in the ThinkStation P620 “BIOS/UEFI” download section of the “Drivers & Software” support page. Another tool is the “The Think BIOS Config Tool”. The latter has a GUI.
The settings I changed are:

  • TBTPortSlot to Auto
  • TBSecurityLevel to No Security

I don’t use the TB3 header on the motherboard. I have the pins jumpered on the AIC to keep it powered. I don’t put my workstation to sleep nor hibernate so keep I keep Force Power enabled and no Go2Sx dancing FTW.
The AIC is in PCIE slot four. It is using NVM23 firmware with the legacy non-DCH driver and software (17.4.77.400). I’ll upgrade the firmware and drivers in the future when I have the time or more likely when a new peripheral requires it.

1 Like

@djgoat - yes, it’s been working fine for over a week now - I store all my data on the external Samsung X5 connected via Thunderbolt, and back it up daily to an old external drive connected by USB. Overall it’s pretty fast and works well.

But I have just seen @ducphuc’s post below, so will try experimenting with the BIOS settings which I couldn’t find before…

1 Like