Return to Level1Techs.com

GPU Firmware Hacking/Reverse Engineering Thread

gpu
firmware

#1

I recently mentioned to someone this thread about GPU firmware signing:

And then today I read about Coreboot/Flashrom devs working on directly supporting GPU firmware flashing:

Coreboot’s Flashrom Working On Radeon GPU Flashing Support
Former RadeonHD driver developer Luc Verhaegen is back at the AMD Radeon GPU reverse-engineering game. He’s now pursuing Radeon firmware flashing with the Coreboot Flashrom utility.

Interesing stuff IMHO, so if anyone else comes across this kind of GPU reverse engineering news or something similar please post it here!


#2

well, as far as i’m informed, the VBios Check is done in hardware early on boot.
It appears as if you can circumvent that, only in linux, by overriding the VBios after it was checked and loaded into RAM, making VBios Mods for Vega possible in Linux.

I assume, AMD Did this to simplify and secure hardware to driver feature-enabling and to distinct Products that are otherwise (nearly) identical.
RX Vega vs Radeon Pro vs MxGPU etc.

So once SR-IOV aka MxGPU support has fully moved to the amdgpu driver, we can hopefully enable it for our RX Vegas


#3

I’m considering that for my VM Host build. I was hoping to put together a XenServer (the Citrix distribution) with the GTX680 -> K2 to specifically use the XenServer-Nvidia vGPU/GPU Hypervisor. But if that technology still locked to the “proper” K1/K2 cards, then I will simply have to defer to using multiple cards in full passthrough.


#4

well, i modified some gtx690s for friend to be grid k2s.
they seemed to still work though he didn’t test them jet. : )


#5

Interesting! Not that I have the experience to tinker with this on my own, but I will happily do testing when someone needs a guineapig.


#6

i’ts actually pretty easy, you don’t even need hot air for that mod.


#7

Well, I even have a hot air station. Nothing is going to stop me :grin:
Maybe you can give me some links/ descriptions? When I get around to it i’ll try it. Sounds like fun!


#8

some info here
http://www.eevblog.com/forum/chat/hacking-nvidia-cards-into-their-professional-counterparts/


#9

Thanks! Did not expect to find something like that on the eevblog forums :sunglasses:.


#10

well, actually Gniff is the one who basically started it on the EEV Blog Forum.
And thanks to all the research he and they managed to do, the gtx690s two cores were recognized as two grid k2s.

i sadly couldn’t find explicit depictions for both cores on the 690 in that thread, but if you google you’ll find them.
And if you shoot me a DM i can send you pictures.
Don’t want to post them here because coppyright etc.


#11

So this was achieved via removing some resistors? Anyone know if it’s still possible on a GTX1080?

Also you mention it’s possible to “Load In” a vBIOS? As in not write it to flash, just load and execute? If this is the case you could have it identify as whatever you like.


#12

More recent firmware has to comply with Secure Boot requirements which means a signing check is mandatory. No one has cracked VBIOSes with secure boot checks yet. This is why there are no Vega or Pascal VBIOS editors.


#13

Yes, that was achieved by changing out a few resistors on the 690.

And i have no clue if it is still possible on any of the newer cards since Reference material is not existent.

You either need high res pictures of both sides of the cards pcb, from your target and the Quadro, or you get the cards to test and probe on them.
Since no one touches quadros to take those pictures, so you have to get one yourself, and that isn’t cheap.

Gnif managed to collect some money and buy the respective gpus in that time to realise that mod.

In the Beginning of the Vega Bios topick i somewhere read that you could load a modded vbios, but only in linux since this is where you have control of what to do at early boot time.
As far as i understood and read at that time, (it might not be correct!), all BIOSe, Main System BIOS and VBIOS get loaded into main System RAM very early at boot from where they are then read and used.

The verification in vega seems to happen before or during that time, allegedly, meaning that once in RAM, you can override that with your own VBIOS, before the driver picks it up.

That is the theory that i still remember researching. Take it with a grain of salt.


#14

So you think it’s a Time-of-check to time-of-use exploit on Vega. Interesting, that would suggest that the verification is done by the driver not the card itself, right? If it was verified by the card, you’d think it would load from system RAM once, and then check and execute from GPU-RAM.

Unless the CPU is allowed to access GPU-RAM? Wouldn’t that break potentially break HDCP, which is part of why we have this mess at all?

According to posts by an AMD employee on the Phoronix forums (see linked post and all following posts by the same user), AMD is contractually obligated not to reveal details about the internals of their GPUs, because those GPUs implement DRM and security by obscurity and the tiresomeness of reverse engineering is apparently part of what Hollywood’s counting on. In other words, AMD may be contractually precluded from revealing the internals of a DRM-free GPU (to enable open source firmware to be developed, for example), because that DRM-free GPU would be so closely related, in terms of internal architecture, to a DRM-implementing GPU that it would provide useful information to people trying to break that DRM.

From: https://www.devever.net/~hl/intelme


#15

Vega contains its own little processor which seems to do the VBIOS check.
The Linux Driver does as far as i know, not verify the VBIOS.

I assume the card checks while loading. At least that is what i have heard.

Thinking about it, you can just patch the driver and not care for the vbios.


#16

From which RAM is Vega doing the verification from though; System RAM or Graphics/GPU RAM?

Unless the GPU RAM can’t be initialized without VBIOS, wouldn’t the process look like this:

  1. GPU reads VBIOS over PCIe
  2. VBIOS is saved to GPU RAM
  3. GPU verifies VBIOS signature
  4. GPU executes VBIOS

#17

It actually doesn’t matter, what matters is the timing, override it after the hardware checked it and before the driver starts using it.

To answer your questions as far as I can, I suggest you start googling or ddging for that stuff.

The VBIOS is located in a SPI EEP? ROM chip on the PCB of the GPU, connected to the SMBus of the PCIe slot and the system.

The normal system bios is also located in a spi eep? ROM connected to the SuperIO controller or the PCH or the smbus or how ever some engineer designed it to be.

The system bios is the first thing to ever be loaded from that rom into the CPU / ram.

I assume, that the GPU is to dumb and unable to initialize itself. I assume, the CPU gets the VBIOS in system ram to do that with the GPU.

From early discussions and AMD material, it seems that the, let’s call him " secure processor" inside the Vega GPU, somehow checks the VBIOS early in the post / boot process.
No clue if he does read it from ROM, from system memory or from GPU memory or from where ever.

Though sure is that It does verify it very very early, way before the kernel of the OS takes control.


#18

I have a suspicion, that maybe you could try voltage glitching the GPU while it is booting. Suddenly dropping the core voltage can cause all sorts of instability in processors, including skipping instructions, allowing you to jump over failed checks. This is very similar to the RGH on the xbox360 or the boot rom dumping hack on the nintendo switch.