How much do you trust vibe-coded software on your home network?

Dev Zone Thoughts on AI
, , ,
1

Something I’ve been thinking about with all these LLM coding experiments. Getting a tool to work on your own machine is one thing. Giving it access to your Proxmox server, Docker containers or NAS is where things start getting interesting.

Say you’ve built a small dashboard with AI assistance. It works, the basic tests pass, and everything looks fine locally. Now you want to use it to manage a few services on your home network. How far would you go with reviewing the code before trusting it?

The whole thing got me thinking about how much work really goes into making AI-generated code reliable. For example, a couple of SaaS companies I collaborated with used a service specifically to clean up vibe-coded applications, this provider called Redwerk. It’s a serious company, so apparently there’s enough demand for this to be a business of its own.

Personally, I’d be particularly concerned about how the application handles credentials and permissions. A bug in a read-only dashboard is one thing, but giving the same application permission to restart VMs or modify files is a different story.

Do you manually review everything that touches your infrastructure, run it in an isolated environment first, or rely on restricting what the application can access? And if you’ve been running AI-generated tools for a while, have you come across any problems that weren’t obvious when you first tested them?

2

I don’t review the code, I review what it touches and the effects of that.

I’ve vibecoded a lot of things, document editors, cooling software to manage my fans and aio, my own TUI harness, wayland compositor, CNN that removes breaths from recordings, etc.

Take my cooling program, sounds smart right? Vibecode some ai slop that manages the literal temperature of my fans and 450$ aio. Good way to have ai slop blow up my computer, the redditors would say.

Did I read the code? No, why would I? I can, but what’s the point? It passed gates, the logic is sound, does what it should and not what it shouldn’t. I’m dumb caveman human who is banging rocks together on computer, talking to high-dimensional latent space that encompasses a vast collection of knowledge. I trust what I built, the ai just wrote the code.

If you build slow, with a plan, things work out fine. If you build fast or in one shot, well, more room for error. Slow is smooth, smooth is fast. But some people like pressing enter and walking away, then coming back 6 hours to finished product. I don’t, i like “rubber ducking” or “pair programming”.

I’m no developer. I’m no programmer, engineer, anything. I’m just a dude who presses enter a lot and asks a lot of questions. I learned building with both a plan and understanding in mind, while slow on the front end, saves time on the back end not having to fix problems. idk, im just a guy, throwing my 1 cents in, im not system network engineer dev dude been doing this for 40 years hand compiled linux with pen and paper in college with the gray beard wizard dude, i just ask robot to make things, with engineering and architecture in mind.

To answer your question, with everything I said, I’d run it till it breaks, then see the damage, what broke, and change it so it doesn’t happen again.

2 Likes
3

I definitely look thru the code pertaining to secrets handling and network calls. Generally, if I can’t follow it, I won’t use it until I do. That’s primarily for anything I use personally. When it’s work related, I generally do a bit more checking of the code (quality + security) before it’s pushed. Having multiple independent LLMs review for security holes definitely can help speed up the review process as well.

I don’t give agents or third party vibe coded projects any access to my personal login credentials for things like email, devices (routers or other PCs), or the like. I know some people do. I’ve heard some say that you can use a password manager to connect an agent to a vault without giving it plaintext passwords, but I haven’t verified if that’s true. Regardless I probably won’t do it anyway.

I’m guessing it’ll vary a lot based on risk tolerance. If you’re comfortable with larger risks, it might be acceptable to do something different than me. I prefer to roll a bit slower to have a better sense of how much trouble I could get myself in first, and to assess if I feel like I could get out easily. If I had more “spare time” I might be a bit more accepting of some risk. Ig for anyone with a more isolated and well monitored environment to run tests inside of, it might not be as large of a risk, but I don’t have that either. I’m also still relatively new to software development, so I take any opportunity to learn from code I didn’t write myself, since it can help me learn the good and bad habits.

4

Agent says in a sandbox, and all code it generates and any changes it wants to make are reviewed by me. Frankly I don’t use AI to manage anything as an on going process. I don’t see the point in having AI ping a IP camera and send me an notification when the camera goes down. I have AI write me a NodeRed flow to do that. I don’t need AI in the loop for daily stuff.

1 Like
5

Not much. That said I don’t trust the median project GitHub very much either, regardless of AI.

6

Medium-ish.

Vibe coded stuff is fine for internal LAN.

On access - yes but only via a shell that is essentially read only. Pointing a coding agent at a proxmox IP and asking it to investigate an issue is super useful. But I’d rather not have it yolo changes on the hypervisor.

More worried about “whoops I accidentally deleted your home directory” than anything malicious though so a thin script barrier is enough

7

It depends what “vibe-coded” actually means.

The heuristics for trust I put into a tool don’t really change if an LLM was used to generate it. I’m looking for a history, reports from reliable sources about it, evidence that it has been tested and is being maintained and so on.

When I “vibe-code” something there’s always a lot of design discussion, validation and questioning. With recent models I trust the result about as much as I’d trust the output of a competent colleague – it can always be wrong and, in particular, it can be wrong because it was under-specified or not tested against reality.

One place I do draw the line is letting agents loose on a system in an unsupervised/unbounded manner. There’s a big difference between using an LLM-generated but testable and consistent tool on a system and letting a probabilistic LLM have the reins while running in a loop.

8

How much do you trust vibe-coded software on your home network?

I simply do not. I’m not a software developer and I don’t have to time to effectively audit the code that’s generated.

If it’s coming from a known quantity, i.e. Jellyfin, then I’m willing give it a benefit of a doubt as people with much more expertise than I have worked on it. I still test ~90% of updates, new software, hardware, on a test system and network for due diligence on my end.

2 Likes
9

Put it in a box. Backup your stuff. Watch it work, set rules and observe how well it follows them. My Proxmox machines back up to a PBS server so if an LLM did nuke the Proxmox nodes, I’d just reinstall and pull all my VMs right back. I do let LLMs ssh in to proxmox VMs after I’ve snapshotted them, and so far they’ve behaved properly. I look at the commands they’re asking to run as well, and make sure they aren’t running destructive ones against poorly defined file paths or anything. Eventually you learn what they can be trusted to do and what they’ll faceplant on.

Tis why I grow tired of big services like Claude locking down their reasoning and other visibility into what the model is thinking/doing. Open models are the future, even something as dead easy as Opencode will show the thinking process, and I’m very often able to stop a model and correct what it’s doing before it even reaches the output stage, simply by seeing it “think” over something I have better information on.

I guess TLDR, watch the models work and see where they trip up. Always back up your important stuff. Read the commands the model asks to run. Eventually you’ll learn what they can handle doing and what they can’t.

They’re also very good at scripting things, so you don’t have to have an LLM (and its possible variation in how it does things because they’re sorta random) in the loop at all.

10

I would prefer to not trust it at all, but the truth is, it is so widely used in the industry now that it is pervasive to the point of being unavoidable.

I am no programmer, but I do occasionally use AI to help me put together some python scripts for managing my servers. I tend to do multiple code reviews with different models at the end and also read through it myself.

While I do not know really any programming language (other than maybe Basic) enough to write code, as long as code isn’t written in an obfuscated way I can generally follow basic logic as long as it isn’t too complicated (which my scripts rarely are)

So, with an adequate level of skepticism I may use some of these scripts, but I don’t use it for any file management purposes. That is just too risky, IMHO.

I have generated on python script that crawls my photo library and sends each photo to a local vision capable model for analysis, and then writes the analysis to a searchable database, essentially creating my own google photos lite without the creepy spying. The script doesn’t get write access to the photos though, and it doesn’t need it.

I have also generated a light little python script that continuously monitors the temperature on the GPU I use for my main LLM in /var/sys and based on the temperature it finds sends a PWM duty cycle command to a USB fan controller to change the fan speed.

It is an enterprise GPU and didn’t come with a fan from the factory, so I have had to improvise, and the fan is quite loud at full tilt, so I have had to limit the noise level wherever possible.

Again, no file system write access, and I have a hardcoded minimum fan speed, so it is less likely to go TOO low and overheat. The driver itself also shuts down the machine if it overheats, so worse comes to worse and it gets things wrong there will be annoying downtime, not dead hardware.

This is the type of stuff I use it for. But that’s about it thus far.

1 Like
11

I just review it all?

I, at the very least, glance through any code my local LLM writes for me. I don’t write anything big though. Python or Java/JS scripts and HTML mostly…. which I can pretty much read very easily so it’s no big deal for me.

Idk that I would trust it to write anything I can’t read on a system I care heavily about, but that’s why I take plenty of backups and only let my LLMs control VMs that are safe to be blown up.

12

Part of the problem is that most people can’t read code to save their lives.

They see something that even resembles the formatting of code, and their eyes glaze over like they just saw a hexadecimal number, and then their brain shuts down.

13

“Vibe-coded” means something different to everyone. I personally trust my little Go Kanban board, which I used LLMs to help write, more than some “popular” Next.js app with hundreds of dependencies just to move cards around. I’d be more worried about one of those dependencies bringing something compromised into my home network.

That doesn’t make my code automatically secure, obviously. But popularity and handwritten code don’t give me that guarantee either. I deliberately try to keep things boring, use the standard library where I can and avoid adding stuff I don’t need. I want something robust that I can still understand.

I use LLMs and agents to accelerate my work. If someone calls that vibe-coding, fair enough. If Libreboot won’t accept a PR because of it, that’s their choice too. I’m no wizard and plenty of people can write better code by hand than I can. For the work I use them for, though, recent models often produce better code than what I’ve seen from colleagues. I still have to read it, question it and understand what I’m accepting.

On your point about “how much work really goes into making AI-generated code reliable”: a shit-ton. AI-generated or not. I’ve seen that in old corporate Java long before LLMs and I’m seeing it again with GoSpeak, my take at an open-source Go “TeamSpeak” I’ve been building over the last year or so. I’m probably slower than people who let an agent build the whole thing and come back when it’s done. I try to read and understand the changes and take them bit by bit. That’s also how I learn.

One thing I’ve really enjoyed recently is letting GLM 5.3 loose on the repo in repeated red-teaming and discovery loops. Keep looking for things to break, question the findings, reproduce them, then go around again. Compared to working on that old Java stuff, it’s much easier now to give my own code that kind of repeated scrutiny without having a whole team available. Everyone else gets to use those tools too, so to be honest I’m still not sure things got easier.

TL;DR: For me it depends on what the software does, what it can access and whether I can understand and maintain it. A Kanban board and something with permission to delete my VMs are very different things to put on my network…

14

I don’t really read at the code, I rely on AI generated unit tests.

You can make the AI generate a dummy Proxmox API and use that for all testing, so no untested code touches the real server.

It does not mean the code is bug free, but trying to read code at the speed AI can write it is just impossible.

If it’s something I need to use on a production server I will test it in the homelab first.