LLM's/Games VRAM Nvidia/AMD Question?

[regulareel]

I hope they can overcome this memory bandwidth limitation and achieve really fast speeds with consumer hardware.

[SlayMoreGames]

It’s not all about the hardware, it’s the software nvidia has the cuda toolkit that all AI system can use/built to use. AMD has ROCm slower and not as plug and play to use, and Intel has Ipex that almost nothing works with but I’m helping intel write hijack code so when a cuda call is made by an AI program intel cards can use it. So AMD could put out a card with 48gb vram but with out the software it would be pointless. And if you think 2k is a lot don’t look at this at 19k NVIDIA Tesla A100 80G Deep Learning GPU Computing Graphics Card OEM SXM4 | eBay

[ElCheapo]

Ollama does support AMD graphics but that doesn’t mean all the models will run on AMD graphics. I do like how when I use a larger model it uses system memory as well. Would make sense to support other vendors.

Ollama now supports AMD graphics cards · Ollama Blog

[Janos]

The 7900XTX is a good alternative.
I have two running in an Epyc Rome system and haven’t found a model that doesn’t work.
These are the models I’ve tested so far.