Training on 7900 xtx

Hardware GPU
,
1

I’ll try to record my notes here for future people, as I just purchased a 7900 for gaming and ML experiments.

After try the Shakespeare example, I Wanted to train a qLoRA, this is a super efficient 4bit adapter that goes with an existing model.

The process only trains 1/10000 of the model weights so you actually do some training at home… without a computer designed to fight god!

To run the training I had to fix an issue with the bitsandbytes python library, thankfully there was a fork with a solution for Rocm, and with that solved managed to start the train with llama7b, let see how it goes in 8hours!

IMG_20230925_084821
IMG_20230925_0848211920×1440 228 KB

1 Like
2

Paused the training and didn’t restart from the checkpoint so starting over today.

I’ll look into batch size and config, before I try this multi decoder project GitHub - FasterDecoding/Medusa: Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

3

What is your experience so far?

4

It’s been solid using the rocm-pytorch container.
And apart from an issue the bitsAndBytes library it’s been going well.

Currently training a model to play Pokemon Red!

2 Likes
5

I am looking at 7900 xt for nearly the same efforts. Would you say the price is worth it. I’m in Canada and it hard to find a 7900 xt for under $1000.
Amd.ca has the 7900 xt for $1172.

6

My recommendation would be to buy the card for gaming first and secondly use it for AI.

I didn’t have the option of second hand 3090’s so take that into account before dropping 1k on a card.

1 Like
7

Looking at my wandb review and I’ve trained 219 hours now.
Since it’s been hard to find details around 7900 training, I’m sharing the training metric’s below.
I hope its helpful in future!

2 Likes