I feel like my relationship with LLMs is strange. I don’t really trust them to do anything, so I rarely use them beyond searching for information and correct grammar.
Yet, I love tinkering with them, enough for me to buy a Intel Arc Pro B70 just to tinker with higher paramater models. I constantly chase the best arguments for llama.cpp to squeeze out maximum tokens per second, even though I never actually use them.
Then I spotted the “MCP Server” section in the llama.cpp interface and wondered what it was. After reading up on it, I thought, “Cool, that sounds like an interesting project idea.”
My go-to language right now is Zig, and luckily it already has everything I need for JSON parsing. Setting it up was a breeze.
As a joke, I originally built a tool that could call another LLM. Only after trying the project opencode did I realize I was essentially building an agent CLI.
It was here that my trust in LLMs and agents grew a bit. I realized the API for MCP differs slightly from the ChatGPT v1 API in how they format tools. I was dangerously close to losing motivation knowing I’d have to refactor the tool list if I wanted nested tool access. I then tried opencode for fun to help refactor that code, allowing me to build tool lists compatible with both standards.
After the refactor, I turned the MCP server into an agent. Knowing how most agent clients are bloated JavaScript codebases, I intentionally kept mine incredibly simple.
It’s still a little buggy and slow, but those are problems for me to solve later.
At the very least, giving local LLMs access to web search and requests has vastly increased their usefulness. In fact, I don’t really feel I need big company APIs anymore.
The project is now in a state where I’m satisfied. I’ve noticed the LLM sometimes struggles to use certain tools correctly, so I’m hesitant to give it direct access to a shell. Instead, I’m building specific tools for certain programs. I’ll likely end up adding a program whitelist and stealing some tool definitions from opencode, since the LLM seems to have no issues using those.
Anyways here is the project if anyone wishes to roast my code or try it out:
Also probably the most boring name for a project ever.
When I say simple I mean this simple, it could probably do with some ANSII colors and the like to make it more readable.
Never really expect anyone to ever use this considering the million other options but it serves my needs and in the end that is all the matters.