answered some of my own questions. it appears ollama does now support AMD!
so I started poking. at first ollama service wouldn’t start
ollama[18396]: Error: could not create directory mkdir /var/lib/ollama: permission denied
so we make the dir and give permissions (after finding the user in the service)
$ sudo mkdir /var/lib/ollama
$ sudo chown -R ollama:ollama /var/lib/ollama/
success! service starts but falls back to cpu. At first glance my 6750XT [gfx1031] looks not supported:
ollama[30982]: time=2024-03-15T17:45:38.102-06:00 level=INFO source=amd_linux.go:88 msg="detected amdgpu versions [gfx1031]"
ollama[30982]: time=2024-03-15T17:45:38.104-06:00 level=WARN source=amd_linux.go:114 msg="amdgpu [0] gfx1031 is not supported by /tmp/ollama1209781978/rocm [gfx1030 gfx1100 gfx1101 gfx1102 gfx900 gfx906 gfx908 gfx90a gfx940 gfx941 gfx942]"
As a test I ran basic ollama run llama2
and it worked but was a little slow…
However we can workaround it by adding this line under [Service] Environment="HSA_OVERRIDE_GFX_VERSION=10.3.0"
to the ollama service: /etc/systemd/system/ollama.service
Restart the service and:
systemctl restart ollama.service
Warning: The unit file, source configuration file or drop-ins of ollama.service changed on disk. Run ‘systemctl daemon-reload’ to reload units.
okay… we’ll reload the reload the daemons then try again:
$ systemctl daemon-reload
$ systemctl restart ollama.service
and… VIOLA! It works:
ollama[44620]: time=2024-03-15T19:43:25.001-06:00 level=INFO source=amd_linux.go:88 msg="detected amdgpu versions [gfx1031]"
ollama[44620]: time=2024-03-15T19:43:25.001-06:00 level=INFO source=amd_linux.go:246 msg="[0] amdgpu totalMemory 12272M"
ollama[44620]: time=2024-03-15T19:43:25.001-06:00 level=INFO source=amd_linux.go:247 msg="[0] amdgpu freeMemory 12272M"
llama2 test is is now significantly faster. didn’t run benchmarks but the difference is night and day.
Great success!
So it looks like the 6750XT is not officially supported but does totally work after applying the HSA_OVERRIDE_GFX_VERSION=10.3.0 work around.
now to try to set up for coding…