RX 9070 vs 5060Ti 16GB for AI usage on Linux

High-Performance Computing Machine Learning, LLMs, & AI
, , , , , , ,
1

Hello!

I would greatly appreciate your advice regarding which GPU I should buy for my needs.

My specs are:

  • Ryzen 5 5600X on B350 mobo
  • 32GB DDR4 RAM
  • Radeon RX 6600 8GB
  • 2x 1440p monitors
  • openSUSE Tumbleweed and Windows 11 (Linux 99% of the time)

Most of the time on my PC I just read in a web browser or i work on virtual machines (either locally or on my Intel NUC “homelab”). I like to game occasionally, I vastly prefer to use Linux as much as possible, Windows 11 is for strictly Windows-only things like software my degree course require to use.

Now I want to learn more AI too, experiment on it locally. My idea is to use n8n or tool like that for homelab automation and while I know tools like Ansible and Terraform exists but I want to try AI to know if it can be one tool that fits it all. I also started using openai-whisper a lot for making notes out of courses I buy for learning various tech topics.

My dilemma is Radeon RX 9700 non-XT vs Geforce RTX 5060Ti 16GB. I don’t want to get the 5070 because of 12GB VRAM and I would highly prefer not to spend 5070Ti money.

I prefer Radeon brand because of amdgpu, more consistent gaming performance on Linux (recent GamersNexus video) and overall dislike for Nvidia, but as an AI newbie I have no idea whether ROCm is good enough for my use case and/or RX 9700 will be fast enough to not be annoying to use.
I don’t mind compiling and tinkering a little to get ROCm support or to use HIP for CUDA emulation (like I do now for openai-whisper) as long as performance will be fine.
I might try image/video generation sometimes but this is far from priority. I know currently Nvidia has big advantage here.

I prefer to buy GPU ASAP before GPU prices will rise. Also, is PCIe 3.0 an issue here? Will I lose performance in AI?

2

The 5060Ti is a good option. It has reasonably fast memory (although narrow in bus size, its fast DDR7) so it competes in bandwidth with 256Bit GDDR6 cards. Nvidia is easier generally to get working, and typically faster in general AI.

The 9070 is useable sure, but for AI I would probably pick the 5060Ti 16Gb.
PCIe 3.0 isn’t really an issue.

3

If you want any kind of speed you need cuda.
If you are only going to setup a simple chat LLM you can go for the amd, it will generate text faster than you can read. But for anything other, nvidia is king, there is simply no way around that.

As for vram. If the model you run won’t fit in vram, it will also slow everything down significantly due to shared memory. And using shared memory with LLM:s is a hit and a miss. Sometimes it works but slowly, sometimes it doesn’t.

So if LLM:s is the prime deciding factor, go for most memory, maybe even an older 30 or 40 series to get more memory/cost.
And if LLM is secondary and you will only play around with pre-trained chat models, go for amd.

This might be an interesting read for you:

1 Like