Radeon VII — gfx906 – Hardware is tested working and found .
So I recently ebay’ed one of these and I’m pulling out my hair trying to get Ollama on Ubuntu 25.04 Linux to use it.
I have failed to get it to work in packages. I tried downgrading to ROCm 5.7 but I think I hit a problem where is doesn’t work past ubuntu 22.04.
I am currently trying to use a docker container but that isn’t building correctly. Using a compose file I found online. Might try my first from scratch compose…
Yeah it was his video about the Radeon VII that encouraged me to ebay this one. His video was on windows but I should rewatch it to see if the ROCm was an issue in windows.
I’ll watch this link also in case it is not the same video.
edit:
I see he mentions Ubuntu 22.04. I think that is my pain point that ROCm 5.7 doesn’t work past this ubuntu version. I’m not clear on the way that would be.
One of the biggest changes between Ubuntu 22.04 and 24.04 is the move away from Xorg in favor of Wayland. Ubuntu 24.04 has started the transition to making Wayland the default, and that may be part of why ROCm 5.7 isn’t working properly.
It’s also possible that no one has updated the PPA packages for 24.04 yet. Have you tried building ROCm 5.7 from source instead?
Totally ignorant here, but I’m gonna suggest checking what drivers you are running, if radeon or amdgpu. The second one is present in newer kernels but not active by default unless it’s a very recent one. I’ve heard the newer drivers improve performance and other aspect of older AMD cards from GCN era. Maybe for Radeon VII, too?
Take what I’ve said with a pinch of salt.
You can try to see if amdgpu drivers are already present on your current installation. If so it is, then you can force them by modifying a config file. I can’t remember which one, though.
I have a deadline coming up, but in a couple of days I can look it up and see if I’m making sense or not. If I am, I will post some instructions to do this in the hope it improves the situation on your machine. But I will be able to do this stuff only in a couple of days
If, instead, you don’t have those drivers, is possible that updating the kernel to a more modern versions (like a more updated Liquorix kernel) can install them for you, and then you can force them at boot.
If you do that, still remember to not replace the old kernel, but keep it aside as a fallback if anything breaks.
Also, before doing any change to the OS installation, remember to backup the hell out of the machine
thanks I’ll update this thread if I figure out anything.
But I believe I have to downgrade my ubuntu which means I have to move my pi-torch first… and I don’t wanna break that until this weekend due to working from home.
I wanted to have pi-torch running in a container anyway (my app server was meant to run docker containers and so far isn’t).
So yeah this weekend I might try moving pi-torch and reinstalling ubuntu at 22.04 and trying to remember to never upgrade it.
Since this machine is being used for work, and there’s a real risk of being down for days while troubleshooting, I think it’s completely reasonable to downgrade to Ubuntu 22.04. Stability matters more than being on the latest release in that situation.
I didn’t realize you were trying to set up ROCm on your work system—if I had, I wouldn’t have suggested compiling from source. Building ROCm is not something I’d want to deal with on a production machine. The risk of ending up with a partial or broken configuration and then spending days troubleshooting is just too high.
Not too familiar with Ollama (I know it’s a fork of Llama.cpp) nor do I have the Radeon VII but can you not use Vulkan with the GPU? It should work just fine or atleast it did for me on Windows with the Vega64.
I have tested and functionally used ollama 12.3 with rocm 6.4.1
47t/s with the ollama combo
Switched to llama.cpp
95t/s with the llama.cpp-gfx906 fattn build on my Radeon VII
Small gguf models, no peak VRAM single model work.
EDIT: Ubuntu 24.04 currently, have tested in 22.04 too, working and used in containers
For anyone needing this the gfx906 series has been added back support.
The Mi250 is the same series of cards that the MI50 and Radeon VII are too, gfx906.
You can install the up to date ROCm 7.10 or 7.11 this way.
Additionally if you are using llama.cpp you can either use the ROCm backend with the drivers mentioned above, but you can also use the Vulkan backend for inferencing only. For the Vulkan backend all you need the is GPU itself, the amdgpu driver and the right flags to compile the software.
I do have some MI50 based cards that I use this way. Works well. ROCm (via AMD’s The Rock) adding back support for these old cards was a nice surprise a couple weeks back though.
I have well 8 Mi50/60 32G in a Gigabyte G292-Z20, they’re not the fastest thing on earth for sure, but you can get 256G vRAM for the price of a single 5090.
If you want to bother it’s relatively easy to get ROCm 7 (compile it yourself) up and running forcing the support for gfx906.
If you want the easy way get these containers: github[dot]com[slash]mixa3607/ML-gfx906
example for running Gemma4 (fits nicely on 1 card, plenty of space for large context, runs @70t/s)
# get the model
mkdir /models/@70thf download unsloth/gemma-4-26B-A4B-it-GGUF --include “UD-Q4_K_XL” --local-dir /models/gemma-4-26b-a4b-GGUF
# run it in a container with optimized rocm 7 and llamacpp for gfx906
docker run -d \
–name=llamacpp-gemma4-26b-a4b-thinking \
–network=host \
–device=/dev/kfd \
–device=/dev/dri \
–group-add video \
–group-add render \
–cap-add=SYS_PTRACE \
–security-opt seccomp=unconfined \
–ipc=host \
–shm-size 64G \
-v /models:/models \
-e HSA_OVERRIDE_GFX_VERSION=9.0.6 \
-e ROCR_VISIBLE_DEVICES=0 \
-e HSA_ENABLE_SDMA=0 \
mixa3607/llama.cpp-gfx906:full-rocm-7.2.0 \
–server \
-m “/models/gemma-4-26b-a4b-GGUF/gemma-4-26B-A4B-it-UD-Q4_K_XL.gguf” \
–alias “gemma4-26b-a4b-thinking” \
–host 0.0.0.0 \
–port 8080 \
–n-gpu-layers 99 \
–ctx-size 32768 \
–cache-type-k q8_0 \
–cache-type-v q8_0 \
–kv-unified \
–cont-batching \
–flash-attn on \
–parallel 4 \
–batch-size 1024 \
–ubatch-size 512 \
–temp 1.0 \
–top-k 64 \
–chat-template-kwargs ‘{“enable_thinking”:true}’ \
–jinja
If you want to go down the rabbit hole of seasoned GPU inference here’s the discord server you’re looking for.
was tempted in getting more of them. Is ROCm 7.x giving better performance? My lama.cpp is using a locked docker base image with my best performing ROCm 6.4.1. If 7.1 is an improvement I’d be willing to try it, the Arch Rocm is always on a version I cannot use, so I dev ROCm in ubuntu. I have a weekend of reading ahead.