Thread for discussing people’s experiences with Hermes agent and their local or hosted llm’s of choice!
2 Likes
So far, i have used 35b parameter qwen 3.5/ 3.6 models with some degrees of success, it can make basic skills pretty good, but so far, using a cloud instance of deepseek V4 cloud through ollama make the skills and doing the research, while using smaller models to inference it is a fun experience! It makes these models so much more cohearent when it has so much better data to refer too!
1 Like
Love the Hermes agent harness, it’s the first “agent” system that actually feels like mine and can do everything I want it to. I used to use Claude Code and some random integrations, but felt it too limiting. Plus upgrading to run qwen3.6-27b locally, and setting up an additional memory provider (in my case self-hosted Honcho) really took it to the next level. Lots of fun third-party, open-source projects out there that integrate with it too (like my TUI of choice, herm).
2 Likes
I was just starting to play with Hermes this weekend. I’m reasonably experienced with self hosting but this was my first foray into local AI.
I have pretty modest homelab hardware at home, so my intention was to use Hermes with cloud LLMs, allowing me to keep my “memory” local and thus allowing me to switch between LLMs as pricing and performance change between anthropic/openAI/etc.
I thought this would be better than getting myself “locked-in” to a vendor if all my data is resting inside Claude’s “memory” of me.
(For clarity, this is not a privacy matter. The goal is not to ensure the cloud vendors do not have access to my data. The goal is to ensure I retain access to my data so that my workflow isn’t locked into a singular vendor.)
UI for Hermes Agent ?
I setup the Hermes TUI on a VM and connected it to a few LLM vendors with API keys. Next step was to set up something to allow me to use and play with Hermes day to day from my mac and my iPhone.
I quickly found out that there are a ton of UIs available for Hermes. Almost too many, leading to lots of time spent researching which is best rather than getting going with using Hermes.
Have others experimented with UIs for Hermes and landed on a common answer for the best “just works” UI ? Seems like most are web-based. I’m going to try a few out this week / this upcoming weekend to see what’s best for me.
1 Like
As I say this, I immediately decided to try out Hindsight over Honcho lol will report with findings. Honcho was just so heavy. If you have big enough local models where the dialectic work and peer modeling actually produce valuable insights, then it’s pretty spectacular. Otherwise, it will poison its own well.
Hermes has a built-in web dashboard service you can run, and it embeds the hermes --tui right into the chat tab. I’ve been using that as needed. I’ve also tried out hermes-webui which is pretty nice. I’d lean towards that for a one-stop “I just want to chat” dashboard. Hermes built-in dashboard is more for management. There’s also TUIs, which are my preferred UI. I also set up Signal for always-available messaging when I’m away from home, but that has significantly more setup cost.
1 Like
I’ve been running Hermes for about 3.5 weeks now after hitting walls with the *claws. It’s been refreshing to not have to fit my reqs into a rigid system of skills or questionable/malicious ones from third parties. I have llama.cpp with prompt proxies in docker on my Asus GX10 currently serving my agents Qwen 3.6 35B-A3B Q5 with 128k context. Hermes is installed and running on my MacBook as I do have it doing some local file organization and reading tasks. Hermes-webui lets me get to it from inside my network and while tunneled in. Really liking it so far!
1 Like
I feel the same way! What parameter count is honcho? So far, I have been doing really difficult tasks with Deepseek v4 cloud, (skill making, deep research, programming or building datasets) then letting smaller local models use what the bigger one has built. I also found the memory constraints where kinda limiting for the “personality" or adaptability so I have quadrupled them with outstanding results so far!
1 Like
I have mine running on my desktop and keep it inactive until I need it. I usually like to see how different models handle radically difficult tasks, and like to see how’s its “personality" of sorts develops overtime. I used to mainly use anything llm on Linux, but it’s Interface is almost too simple (I like options but not too many, but anyllm seems sparse even for my tastes if that makes any sense) for me. I am an oddball and mostly interact though it through its terminal ui for now, but occasionally check tabs on its web ui. I think hermes will be my permanent replacement until the next best things comes along. I’m glad you are also having a positive experience so far!
1 Like
Is anyone installing hermes with helm? I was going to try this helm chart - GitHub - ultraworkers/hermes-agent-helm-chart: The community-driven unofficial chart packages Hermes Agent for Kubernetes with cloud-native defaults, explicit state-safety guardrails, and flexible composition points for platform teams. · GitHub
I am just starting with AI with a budget built Asus pro WS w790e. AS I have not worked on server stuff before and the last time I used a terminal was in the 80’s on a VAX server. Setting up the Bios was not fun. Installing Linux disro getting all the devices to work finding copy and paste on the terminal did it work restart a million times Troubleshooting just getting started up. Install Run Hermes Locally, Ollama, Gamma 4:31b, Configure Hermes, Optimize for Speed, GPU Offloading. Now I have to learn the dashboard. Then find out how much power costs for 2350 watts 24/7 a mouth. What fun I am having. Oh-no crash what when wrong? Don’t use the toaster.
1 Like
NeuromancerXR that Honcho uses is supposedly a fine-tune of Qwen3-8B. Small tailored models can outperform larger generalist in resource use, specificity of task, and wall time. More important to me was reading about their approach here Introducing Neuromancer XR ; both fascinates and appeals to me. That article inspired me to attempt replicating that model; I expect to fail and learn a lot along the way. I’ve not read much about the MR or other models.
I’ll take a look. I read and read and C&P and hope to get it working and figured out what I did later. I am on day 2 now just learning how link my gaming i9-14900KF 128GB ddr5 ram computer and 10 x 4 core 16 GB and 32 GB ddr4 laptops to my W790e WS server with exo-explore/exo. To load a 370b and larger AI’s on my own hardware.
Oh my EXO Prerequisites: this is going to fun.
It’s been mostly positive, yeah. I’ve got one agent doing some creative fiction writing and with 128k context window am currently having an interesting, sorta rough time with context compaction killing the chat. So much so that I can’t even get to the questions asked by the agent. Finding ways around it but forcing it to save to .md files at every iteration and mod done are helping combat this quirk.
Next steps are trying to figure out how to hand off tasks between agents! I think I’m enjoying the learning more than anything at this point.
I went with the self-hosted Honcho setup and chose to use local LLMs for all honcho workloads. I split it across smaller <9b models for small/quick stuff, and my main qwen3.6-27b for the heavier dialectic stuff. It worked, sure, and I imagine if you had dedicated, large models for all those workloads it would perform really well. I for one refuse to use cloud models for something so personal lol Its peer building and “watcher” style of constantly processing sessions is handy for near real-time memory feedback. If it’s working well, it’s impressive, but there’s a lot of bloat and maintenance debt for self-hosted.
Hindsight has been alright, but definitely not as powerful as honcho. Comes with pre-loaded transformer and embedding models, and only uses an LLM for specific retrieval paths, if wanted. Significantly less overhead, but you have to really nudge it to consistently use it. I’m wondering if there’s some hooks I could set up to “auto-retrieve” on session start 
That sounds like alot of fun, and I love that idea! Making a cluster of machines to spread the workload to run the large models sounds like a very interesting idea, I wonder if I’d be able to do the same with some old gaming laptops I find on eBay or Facebook Marketplace?
I’ve been looking into the hooks, I want to inject a lot of different information automatically.
You can get Hermes to write you plugins for Hermes, which is kind of funny, and you can get it to automatically modify the system prompt.
It might be easier just to add the instruction to the AGENTS.md file, and just tell Hermes what you want it to do.
One problem I have been having is when you want to modify each prompt, some parts of the prompt you can’t just modify because of the KV cache. You can modify the user part of the prompt, but that seems to force the AI to take action, you can’t really use it to give the AI a gentle nudge.
I switched from using Openclaw to Hermes a few weeks ago and I’ve been enjoying it.
Hermes seems a lot more polished than Openclaw and i had noticed that my openlaw instances had got really slugish.
I’m fortunate to have access to a couple different openAI pro subscriptions that i use with my agents and its really good with that.
I did dabble with running my own model locally but i do not have enough hardware to really run a model big enough with enough context length to really make it usable.
I run my agents on some old corporate cast off mini pcs running Ubuntu server and they seem to work really well there. Nice thing about this setup is i can let the agent do whatever the hell it likes on that box and if it all goes pear shaped i can just wipe it and start over.
To interact with them, i either SSH in and use the TUI or ill connect an SSH tunnel and use the Hermes Dashboard webui with its embedded TUI enabled.
I used to use OpenWebUi to interact with the openclaw agents, got them to build that capability for me just by prompting them and installed what they asked for on the machines running the agent. But with hermes ive been happy enough with the TUI and webui that comes with it so far.
I’ve used Heres with OpenWebUI, and it doesn’t really work.
You can’t see when the AI is making tool calls, but you can kind of get around this by telling it to make a special note every time it uses a tool call.
The biggest issue was the call confirmation, there is no way to agree or deny in the webui. You can use the smart confirmation setting and allow the AI to do all calls that are considered safe, but it’s far from ideal.
The revolving door issue I see with agent harnesses like Hermes and OpenClaw, is exactly why I built my own. Interestingly, Pewd just dropped a near replica of what I use daily in house(We just actually use opencode instead of porting it) essentially a LLM workspace of tooling and systems to do what these autonomous harnesses aim for.
The level of paralell construction “behind the scenes” in the open source world is astounding. The irony being most of the software made are to use the agents that make the software that…
I vibe-coded an offline first, deterministic harness with isolation everywhere and made it fully auditable. It’s a niche harness, it’s made around tasks, there is no chat. I actually have claude use it to complete local LLM work i.e. frontier provider orchestration and planning.
I’ve been bugtesting it so long Claude and I have built a handful of tools that I use frequently, within harness. It also uses a bunch of API’s, a handful of local inference methods, and for the most part, is pure rust.
I’d love to see a comparison of the MANY harnesses out there. Zot caught my eye the other day, but Pewd’s Odysseus is likely better fitted for Average Joe.
1 Like