Hello all,
I have just purchased a Framework desktop board. My grand plan is to set up a Homelab that can run AI workloads. I have been trying to set it up for the last couple of weeks but have not had any good progress. I will be honest my experience at this stuff is not the best. I have been running Arch Linux and local AI for about a year now. I would say my current level is a Script Kiddie but would like to learn to get out of that phase. Below is my hardware, software, and dead ends i have run into.
-
Hardware
-
Framework Desktop
- Strix Halo AI Max 395+ 128GB
-
-
Software
-
OS
-
Arch Linux
- I am using the computer as a desktop right now.
-
-
Potential AI stack
-
Ollama-ai server
-
Openwebui-gui front end
-
Comfyui-for text to image generation
-
n8n-for ai agents
-
zed-for coding agents
-
docker-making all this a container sounds like a good idea
-
-
-
Previous failed AI stacks
-
-
reasons I could not get it to work
-
could not get rocm installed
-
ROCm Installation guide on Arch · GitHub
- step 3ish when i had to use vim i got lost
-
-
-
-
GitHub - kyuz0/amd-strix-halo-toolboxes
-
reasons I could not get it to work
- was able to get it to run chatgpt oss 120b would just hang. I think i need to use ollama anyway to allow me to integrate with zed but i could be wrong.
-
-
https://github.com/kyuz0/amd-strix-halo-comfyui-toolboxes
-
reasons i could not get it to work
- got it running but took a really long time to make an image. so i turned it to 96gb of vram and it would just crash. I think this is also a dead end because i think i want to have comfyui work through the openai gui. I may be completely wrong on this.
-
-
I do hope all of this was coherent. If anyone could point me in the right direction to set this up I would be most appreciative. I believe a docker or some other container would be the best for this project. I don’t know how to code to code but couldn’t really find anything that made since to me. Thank you for any help.
