First Linux Adventure: Intel Arc LLM stack advice needed

High-Performance Computing Machine Learning, LLMs, & AI
, ,
1

Hello Level one techs!,

First post after a week diving headfirst into the world of local LLMs and Linux.

I recently bought an Intel Arc B580 and, with absolutely zero prior Linux experience, decided Ubuntu was the way to go after seeing it mentioned in so much Intel GPU and LLM documentation.

I was going in pretty blind after having a little play about on windows with an older nvidia card. While I had intels ai playground running on windows from what I could tell linux was the way to go for local models.

To say I made a lot of mistakes would be an understatement. I reinstalled the OS more times than I can count. I had no idea what I was doing, I tried google gemini on my phone and ended up with a cheap trial for pro. With Gemini holding my hand we started steaming ahead both hallucinating a bit maybe.

At various points, I was running outdated Linux kernels because I thought I absolutely needed to use IPEX-LLM and outdated Ollama versions. I’ve only just recently hopped onto Ubuntu 26.04. Gemini was incredibly stubborn about keeping me on 24.04, but I think that was just because of the older documentation I was feeding it. We’ve been falling upward and got a big mixtral MoE model running and trying to code a vram monitor while commenting the code in a pirate accent as a benchmark.

Right now, I’ve managed to get Ollama running using the SYCL/Vulkan backend, but I have a strong feeling this isn’t the optimal way to squeeze the best performance out of this card.

Despite the friction, I’ve learned an immense amount through trial and error. Gemini has been a fantastic tutor for learning CLI commands and general Linux . even if its decision-making falls apart when the person prompting it doesn’t know what to ask. And I’m possibly picking up bad habbits who knows but having something interactive that I can throw errors into and have it explain the issue and solution not just spoon feed me prompts (but also spoon feed me prompts)

My ultimate goal is to have a highly capable, fully local AI assistant to help me self-host and learn. The progress so far has been incredibly rewarding:

The Homelab: This weekend, I finally revived a 4-bay Datto mini-server a friend gave me a year ago. I used to get completely lost trying to set it up, but now it’s running a bunch of Docker services and backing itself up nightly.
Learning to Code: I’ve been using gemini to learn programming. Instead of an online course, I’ve had the AI build a curriculum tailored to my exact interests. All my HTML, CSS, and JS practice examples are themed around wildlife photography and a new camera I recently bought.
Vibe coding: I made a fun task scheduler vibe coding with google antigravity. It takes a text dump of my tasks, checks the local weather forecast, and automatically assigns indoor or outdoor tasks to open slots in my Calendar and spaces out labour intsenive tasks.

My Question for the Community

I feel like I finally know enough to start asking the right questions.

Given an Intel Arc B580 running in an older gaming PC**, what software stack would you recommend from the ground up? Specifically, I’d love advice on:

  1. OS/Kernel: Is Ubuntu 26.04 fine, or is there a distro/kernel better suited for Intel Arc compute like catchyos?
  2. Backend/Drivers: What is the best way to leverage the B580’s hardware (e.g., sticking with SYCL/oneAPI, IPEX, or something else entirely)?
  3. Frontend/UI: What are your favorite user interfaces for interacting with local models?

Sorry for the wall of text I’ve been speaking to Gemini for a week straight about this all. Thanks for any help.

2 Likes
2

Hello!

I’m in a similar boat to yours. While my rant might not be helpful, I hope it atleast makes you feel less isolated in the chaos of running Local LLMs on an Intel Arc GPU. For context, I’m running Ubuntu 24.04.4 LTS on an Intel b580, Ryzen 5 7600 & 32Gib RAM.

I’ve got PTSD with Jira tickets so all work struggles have been logged in markdown files in this repo: Github - spatel333/ollama-on-intel-arc …as well as some sources for how to prep the GPU overall.

My biggest impetus for improving the stack came from discussions around OpenVINO have superior performance to Vulkan.

sources:

  • Compelling Bytes’s Youtube Video: “OpenVINO Changes Everything for LLMs…”
  • bdhcuidbebe’s comment thread in YCombinator Blog Post 2025: “Llama.cpp supports Vulkan. why doesn’t Ollama?”
  • SUPEROXIDES Level1Techs Blog Post - April 2026: “Llama.cpp for OpenVINO. How does a half-baked build stack up against Vulkan?”

While a basic ollama deployment worked on my machine & even an OpenUI GUI could be spun up, the idea of subpar performance just didn’t sit right. Why invest in this hardware if we’re not getting its full potential? So I set out to learn the stack & where OpenVINO would fit.

Similar to you, I’ve leveraged Claude to help demystify how the software meets the hardware. The general stack it mapped out is this:

Concept Layers Description Column 3 Column 4
Framework Here is where OpenVINO, PyTorch, oneAPI/SYCL, and other toolkits sit. There is no direct communication with the GPU, these simply call down to the API level.
Userspace API Implementation Here is where Vulkan, OpenGL, VA-API, Level Zero, and other libraries sit. This is where libraries run API calls to leverage GPU compute. They don’t technically speak with the GPU directly, they speak with the Kernel driver which in turn speaks to the hardware registers.

Different stacks leverage different libraries & compilers (I’ll touch on the libigc1/libigc2 struggle later in this post) | | | |
| DRM / libdrm | The Kernel driver gets exposed to the userspace as device nodes (e.g. /dev/dri/card0). Everything above this is in userspace. | | |
| Kernel Driver (KMD) | This is the only piece that talks directly to hardware registers. It receives system calls [such as ioctl()] and works on the registers. | | |
| Hardware | The GPU itself | | |

A key note was in the Userspace API Implementation section: “different stacks leverage different libraries & compilers”. This would explain why OpenVINO installation steps recommended Intel Drivers (Source) which leveraged libigc1, while Ubuntu 24.04 LTS ships with libigc2. A blocker which highlighted the gap in Intel Documentation.

In working on an ollama deployment, I’ve noticed the IPEX-LLM repository is archived (link to repo) and having an OpenVINO backend does not seem to be a feature of ollama. The path forward may be another product such as LM Studio, but I still have much to learn.

My end state is being able to locally serve an LLM for chatbot tutoring, VS Code integration, and ideally work with Qiskit.

If you’ve made progress, do share & if any of my details are misguided or wrong please correct me. This has all been a learning process.

3

The highest performance I have been able to get out of an Arc B70 has been achieved by ditching Linux and installing Windows. Not only is it the highest performance, it takes 10 minutes to set up with no fiddling in comparison to the entire days spent on everything else. I’ve never managed to get anything other than Vulkan llama.cpp working under Linux and the performance is not great when it does. I’m sure there is a lot that I am missing out on, but that has been my experience so far. Make sure to use the pro drivers if you use Windows as it actually matters quite a bit.

4

I’m not an ARC user, so I can’t really say that I’m an expert here, but if it is anything like AMD then I’d suggest that you ditch Ubuntu in favour of something that keeps up with the times a bit more steadily and frequently. I’m using OpenSUSE Tumbleweed myself, as it provides a good balance between stability and currency, but of course you may have different preferences.

5

From both of you @Strawberry @Onno I take the it that Ubuntu is just not where to be. At the risk of breaking the “no discrimination” rule: There’s no shot I’m installing Windows, sorry.

This machine serves as a cross among: personal workstation / gaming machine / homelab education. For this would you still recommend OpenSUSE or anther distro?

6

I won’t recommend a distro outright, but a distro which is updated frequently while, as far as that’s possible for a fast releasing distro, is being tested is something you should be looking for.

I will explain how I came to my choice for OpenSUSE TW (OSTW) for my use care. I migrated from Ubuntu 22.04 LTS to OSTW for my desktop ca 2023 because games kept running into issues with out of date display drivers etc. OSTW is a rolling distro, which means it releases the packages almost as fast as they’re released by the upstream authors, whereas release based distros only bring updates for security and major bugs, but generally not new features or new upstream releases.A downside of a rolling distro is that it generally means stuff breaks more often. However, OpenSUSE also maintains the Open Build System which does do a little bit of testing before it goes out of the door. So it isn’t as unstable as some other rolling distros out there. That works for me on the desktop. For my servers I mainly run Debian, which I’ve been using for self-hosted servers for about 15 years now. The reason for me to then install OSTW on my inference server was that Debian wasn’t up to date enough. So I went for the thing I knew best that was up to date enough and which I knew well enough to not spend a lot of time relearning how to use the OS.

And that is the major caveat when switching distributions. Inside a distro family you’ll likely know the way. E.g. Ubuntu is a debian derivative and shares a lot of the tooling with that distro. There are some notable differences, but it’s not a large jump if you switch between the two. OpenSUSE is from a different distro family, so that means you’ll have to learn a bit more on how the tooling for that particular distro works before you become proficient with that OS. RHEL is yet another distro family. If you’re new(ish) to linux, then sticking to options which have a lot of guides etc can be beneficial. For more obscure options you’ll have to do more digging or befriend some users who can help you get started. In the case of OpenSUSE that means their forums and their Matrix space.

In the end it is up to you to find out what suits you best. Look around and see what fits your needs and knowledge. However, like I said before, given the speed of developments in this space, you need something which updates frequently and steadily, as to not be caught off guard when the next release of your model runner needs some state-of-the-art package if it is to work properly.