GPU considerations for Local LLM/AI

Hardware Hub GPU
, , ,
1

Greetings everyone!

I have probably the most common GPU question around… Which GPU should I get?

I have made myself retarded searching the internet for weeks trying to find a straight answer but I seam to be wondering through a sea of BS and Obsess Profits all strategically designed to guarantee I waste as much of my money as many times a possible. So here we go…

I have built a Machine for running offline LLM and AI Generative stuffs. It has a Threadripper, it has 64megs of RAM, and it has a 3090 Founder’s Edition GPU.

I have found this to be incredibly limiting as I am now expanding into video generation. I believe I need a new GPU because the limiting issue is VRAM. “You do not have enough VRAM to do that”

My Budget is 5K. Can anyone explain to be the real distance between say 2x 5090’s, a NVIDIA Tesla A100 Ampere 40 GB, a H6000 etc…

Thank you so much, I need to get as much video as I can… Time is not the issue, ability to function is.

Thank you

1 Like
2

If time is not the issue, but ability to function is, may I point to ComfyUI, which allows partial CPU offload for virtually all models? Almost no more VRAM issues within a limit, except with performance.

3090 with 24GB is already more than what a great many people are running them with.

3

This topic was automatically closed 273 days after the last reply. New replies are no longer allowed.