Radeon V620 bargain or not worth it?

Hardware Hub GPU
,
1

I’m a EE student who was wanting to run larger models on the hardware I have currently. Don’t have much money to spend on something like a 3090 (they are 1500 CAD where I am) being a student and all. I already have a gaming pc with a 7900xt and was thing about running another gpu with bifurcation for bigger models. The best value things that I was able to find were the Mi50 and V620 considering I already have an amd GPU. The v620 is cheaper on ebay but if a sxm2 V100 adapted to pcie would be better idk. Intuition says that the mixed architectures and having to run on vulkan would be pretty bad for performance. Just looking for some advise. Thanks

2

I’m in a similar spot with an existing high-end AMD card and the cost of adding a second GPU, so the V620/Mi50 tradeoff is useful to see discussed. The Vulkan and mixed-architecture concerns are exactly what make this harder than just adding cheap VRAM; real-world ROCm support and whether the card can share a model cleanly seem like the deciding factors.

3

Yeah there are some P40s with 24gb of GDDR5 near me that I can get for pretty cheap, but those are some really old cards and the mixed architecture puts me off like you said. Don’t know if it would be better to get a new consumer card with 16gb of vram, but then that would limit the size of models I can run. Can’t decide if I want to run better models slower or smaller worse models faster.

4

I’ve got two v620 running in an IBM S822LC 8335-GCA. For me, it’s worth it. ~60tgs and over 300pp reliably. I’m using llama.cpp and vulkan. rocm is too much of a headache for ppc64le so it works for my use case.

5

Thanks for your insight. Its good to know that the performance is pretty good. I’m probably gonna order one from an ebay seller that has a return policy, and see how it works in my case. If it makes sense ill keep it. There is a CMP 170HX seller near me with confirmed 64gb of hbm2e so if they give me a really good price i might go for that considering the higher density of vram; since I wont be able to run more than 2 cards, because of the lack of more pcie lanes for am4. Also the performance of the 170HX is way higher but it also costs a bit more than double the v620.

6

I managed to get the ROCm stack running on my machine. Using a fork of Strata I’m now getting ~60-100tgs and 1000-1600pp

1 Like
7

Do mind elaborating on which strata fork? After experimenting with strata at work I see a light to use strata on my AMD desktop but the vanilla strata does not support my current amd gpu.