Cheap used GPUs for multi-GPU hobby ML rig

so the mi25 might be towards the end of rocm support, its technically not supported anymore but still works on at least 5.2
pytorch recently updated to 5.4.2 so I need to test that

cuda just works its kinda wild how easy it is
with rocm you need a hyper specific kernel version for it to install the driver completely and correctly which can be found on amd docs

the docs keep switching portals so finding documentation is sometimes a pain

if you aren’t a linux jedi master (I’m not) the hyper specific kernel can keep you on really old OS versions like 20.04.4 for rocm 5.2

as someone who has dabbled in both sides I absolutely don’t get why people say Nvidia linux drivers are trash when AMD exists

I made a guide for mi25

not sure I literally learned how to use linux and rocm about 4 weeks before I made that guide

stable diffusion has been

so A4000 is just below the line where nvidia stops gutting cards of features, its FP16 isn’t unlocked and its double percision is gimped

I am not sure how it compares in real world against a titan RTX

it’s about 175$ more at 675$ and has 24GB and a load of tensor cores, but supposedly gimped double percision

3090 is also about the same price at 680$ and stronk

here is a handy AI benchmark site to compare