How much do you trust vibe-coded software on your home network?

Something I’ve been thinking about with all these LLM coding experiments. Getting a tool to work on your own machine is one thing. Giving it access to your Proxmox server, Docker containers or NAS is where things start getting interesting.

Say you’ve built a small dashboard with AI assistance. It works, the basic tests pass, and everything looks fine locally. Now you want to use it to manage a few services on your home network. How far would you go with reviewing the code before trusting it?

The whole thing got me thinking about how much work really goes into making AI-generated code reliable. For example, a couple of SaaS companies I collaborated with used a service specifically to clean up vibe-coded applications, this provider called Redwerk. It’s a serious company, so apparently there’s enough demand for this to be a business of its own.

Personally, I’d be particularly concerned about how the application handles credentials and permissions. A bug in a read-only dashboard is one thing, but giving the same application permission to restart VMs or modify files is a different story.

Do you manually review everything that touches your infrastructure, run it in an isolated environment first, or rely on restricting what the application can access? And if you’ve been running AI-generated tools for a while, have you come across any problems that weren’t obvious when you first tested them?

I don’t review the code, I review what it touches and the effects of that.

I’ve vibecoded a lot of things, document editors, cooling software to manage my fans and aio, my own TUI harness, wayland compositor, CNN that removes breaths from recordings, etc.

Take my cooling program, sounds smart right? Vibecode some ai slop that manages the literal temperature of my fans and 450$ aio. Good way to have ai slop blow up my computer, the redditors would say.

Did I read the code? No, why would I? I can, but what’s the point? It passed gates, the logic is sound, does what it should and not what it shouldn’t. I’m dumb caveman human who is banging rocks together on computer, talking to high-dimensional latent space that encompasses a vast collection of knowledge. I trust what I built, the ai just wrote the code.

If you build slow, with a plan, things work out fine. If you build fast or in one shot, well, more room for error. Slow is smooth, smooth is fast. But some people like pressing enter and walking away, then coming back 6 hours to finished product. I don’t, i like “rubber ducking” or “pair programming”.

I’m no developer. I’m no programmer, engineer, anything. I’m just a dude who presses enter a lot and asks a lot of questions. I learned building with both a plan and understanding in mind, while slow on the front end, saves time on the back end not having to fix problems. idk, im just a guy, throwing my 1 cents in, im not system network engineer dev dude been doing this for 40 years hand compiled linux with pen and paper in college with the gray beard wizard dude, i just ask robot to make things, with engineering and architecture in mind.

To answer your question, with everything I said, I’d run it till it breaks, then see the damage, what broke, and change it so it doesn’t happen again.

2 Likes

I definitely look thru the code pertaining to secrets handling and network calls. Generally, if I can’t follow it, I won’t use it until I do. That’s primarily for anything I use personally. When it’s work related, I generally do a bit more checking of the code (quality + security) before it’s pushed. Having multiple independent LLMs review for security holes definitely can help speed up the review process as well.

I don’t give agents or third party vibe coded projects any access to my personal login credentials for things like email, devices (routers or other PCs), or the like. I know some people do. I’ve heard some say that you can use a password manager to connect an agent to a vault without giving it plaintext passwords, but I haven’t verified if that’s true. Regardless I probably won’t do it anyway.

I’m guessing it’ll vary a lot based on risk tolerance. If you’re comfortable with larger risks, it might be acceptable to do something different than me. I prefer to roll a bit slower to have a better sense of how much trouble I could get myself in first, and to assess if I feel like I could get out easily. If I had more “spare time” I might be a bit more accepting of some risk. Ig for anyone with a more isolated and well monitored environment to run tests inside of, it might not be as large of a risk, but I don’t have that either. I’m also still relatively new to software development, so I take any opportunity to learn from code I didn’t write myself, since it can help me learn the good and bad habits.

Agent says in a sandbox, and all code it generates and any changes it wants to make are reviewed by me. Frankly I don’t use AI to manage anything as an on going process. I don’t see the point in having AI ping a IP camera and send me an notification when the camera goes down. I have AI write me a NodeRed flow to do that. I don’t need AI in the loop for daily stuff.

2 Likes

Not much. That said I don’t trust the median project GitHub very much either, regardless of AI.

Medium-ish.

Vibe coded stuff is fine for internal LAN.

On access - yes but only via a shell that is essentially read only. Pointing a coding agent at a proxmox IP and asking it to investigate an issue is super useful. But I’d rather not have it yolo changes on the hypervisor.

More worried about “whoops I accidentally deleted your home directory” than anything malicious though so a thin script barrier is enough

It depends what “vibe-coded” actually means.

The heuristics for trust I put into a tool don’t really change if an LLM was used to generate it. I’m looking for a history, reports from reliable sources about it, evidence that it has been tested and is being maintained and so on.

When I “vibe-code” something there’s always a lot of design discussion, validation and questioning. With recent models I trust the result about as much as I’d trust the output of a competent colleague – it can always be wrong and, in particular, it can be wrong because it was under-specified or not tested against reality.

One place I do draw the line is letting agents loose on a system in an unsupervised/unbounded manner. There’s a big difference between using an LLM-generated but testable and consistent tool on a system and letting a probabilistic LLM have the reins while running in a loop.

How much do you trust vibe-coded software on your home network?

I simply do not. I’m not a software developer and I don’t have to time to effectively audit the code that’s generated.

If it’s coming from a known quantity, i.e. Jellyfin, then I’m willing give it a benefit of a doubt as people with much more expertise than I have worked on it. I still test ~90% of updates, new software, hardware, on a test system and network for due diligence on my end.

2 Likes

Put it in a box. Backup your stuff. Watch it work, set rules and observe how well it follows them. My Proxmox machines back up to a PBS server so if an LLM did nuke the Proxmox nodes, I’d just reinstall and pull all my VMs right back. I do let LLMs ssh in to proxmox VMs after I’ve snapshotted them, and so far they’ve behaved properly. I look at the commands they’re asking to run as well, and make sure they aren’t running destructive ones against poorly defined file paths or anything. Eventually you learn what they can be trusted to do and what they’ll faceplant on.

Tis why I grow tired of big services like Claude locking down their reasoning and other visibility into what the model is thinking/doing. Open models are the future, even something as dead easy as Opencode will show the thinking process, and I’m very often able to stop a model and correct what it’s doing before it even reaches the output stage, simply by seeing it “think” over something I have better information on.

I guess TLDR, watch the models work and see where they trip up. Always back up your important stuff. Read the commands the model asks to run. Eventually you’ll learn what they can handle doing and what they can’t.

They’re also very good at scripting things, so you don’t have to have an LLM (and its possible variation in how it does things because they’re sorta random) in the loop at all.

I would prefer to not trust it at all, but the truth is, it is so widely used in the industry now that it is pervasive to the point of being unavoidable.

I am no programmer, but I do occasionally use AI to help me put together some python scripts for managing my servers. I tend to do multiple code reviews with different models at the end and also read through it myself.

While I do not know really any programming language (other than maybe Basic) enough to write code, as long as code isn’t written in an obfuscated way I can generally follow basic logic as long as it isn’t too complicated (which my scripts rarely are)

So, with an adequate level of skepticism I may use some of these scripts, but I don’t use it for any file management purposes. That is just too risky, IMHO.

I have generated on python script that crawls my photo library and sends each photo to a local vision capable model for analysis, and then writes the analysis to a searchable database, essentially creating my own google photos lite without the creepy spying. The script doesn’t get write access to the photos though, and it doesn’t need it.

I have also generated a light little python script that continuously monitors the temperature on the GPU I use for my main LLM in /var/sys and based on the temperature it finds sends a PWM duty cycle command to a USB fan controller to change the fan speed.

It is an enterprise GPU and didn’t come with a fan from the factory, so I have had to improvise, and the fan is quite loud at full tilt, so I have had to limit the noise level wherever possible.

Again, no file system write access, and I have a hardcoded minimum fan speed, so it is less likely to go TOO low and overheat. The driver itself also shuts down the machine if it overheats, so worse comes to worse and it gets things wrong there will be annoying downtime, not dead hardware.

This is the type of stuff I use it for. But that’s about it thus far.

1 Like

I just review it all?

I, at the very least, glance through any code my local LLM writes for me. I don’t write anything big though. Python or Java/JS scripts and HTML mostly…. which I can pretty much read very easily so it’s no big deal for me.

Idk that I would trust it to write anything I can’t read on a system I care heavily about, but that’s why I take plenty of backups and only let my LLMs control VMs that are safe to be blown up.

Part of the problem is that most people can’t read code to save their lives.

They see something that even resembles the formatting of code, and their eyes glaze over like they just saw a hexadecimal number, and then their brain shuts down.

“Vibe-coded” means something different to everyone. I personally trust my little Go Kanban board, which I used LLMs to help write, more than some “popular” Next.js app with hundreds of dependencies just to move cards around. I’d be more worried about one of those dependencies bringing something compromised into my home network.

That doesn’t make my code automatically secure, obviously. But popularity and handwritten code don’t give me that guarantee either. I deliberately try to keep things boring, use the standard library where I can and avoid adding stuff I don’t need. I want something robust that I can still understand.

I use LLMs and agents to accelerate my work. If someone calls that vibe-coding, fair enough. If Libreboot won’t accept a PR because of it, that’s their choice too. I’m no wizard and plenty of people can write better code by hand than I can. For the work I use them for, though, recent models often produce better code than what I’ve seen from colleagues. I still have to read it, question it and understand what I’m accepting.

On your point about “how much work really goes into making AI-generated code reliable”: a shit-ton. AI-generated or not. I’ve seen that in old corporate Java long before LLMs and I’m seeing it again with GoSpeak, my take at an open-source Go “TeamSpeak” I’ve been building over the last year or so. I’m probably slower than people who let an agent build the whole thing and come back when it’s done. I try to read and understand the changes and take them bit by bit. That’s also how I learn.

One thing I’ve really enjoyed recently is letting GLM 5.3 loose on the repo in repeated red-teaming and discovery loops. Keep looking for things to break, question the findings, reproduce them, then go around again. Compared to working on that old Java stuff, it’s much easier now to give my own code that kind of repeated scrutiny without having a whole team available. Everyone else gets to use those tools too, so to be honest I’m still not sure things got easier.

TL;DR: For me it depends on what the software does, what it can access and whether I can understand and maintain it. A Kanban board and something with permission to delete my VMs are very different things to put on my network…

1 Like

I don’t really read at the code, I rely on AI generated unit tests.

You can make the AI generate a dummy Proxmox API and use that for all testing, so no untested code touches the real server.

It does not mean the code is bug free, but trying to read code at the speed AI can write it is just impossible.

If it’s something I need to use on a production server I will test it in the homelab first.

1 Like

That’s also the same approach I take.

Hammer on the code, over and over and over.

Until the model starts returning “idk, the variables you use could use better naming conventions”. Then switch models, and start hammering again. As different models view code from different perspectives I’ve noticed. Framework and architecture vs output kind thing, idk im no jr dev.

I can’t write code, but one thing i’ve always been good at in my life is building (I banged a hammer instead of a keyboard). Do i need to understand the sheer strength of a standard 16d nail to know how to properly utilize it so I can put together a window package when framing a home? I’m not smart enough to answer that question, i just banged a hammer, but I can tell if the bucket of nails bossman threw into my truck is the good brand or not on the first hammer strike.

But i guess that lumps me in with all the other people, i get access to ai, and suddenly i think me some grand Apple engineer, worthy of building monuments.

2 Likes

Yeah, I think the unsupervised loop is where I would get most uncomfortable too.

If I was giving something access to my home network, I would want to keep its permissions pretty limited and have decent logging around whatever it does. That way if it starts behaving strangely, you have a better chance of spotting it early and working out what happened.

The under-specifying point is a good one too. I can see that causing problems even when the generated code itself looks fine.

You’ll want to have it “adversarial” (try and find issues) review its tests, or ideally have another model that may spot issues. If there’s a problem with a test it’ll “pass” while still being broken. Recent example is an app reading a directory as a file (and thus returning nothing), and the test for it was making a fake file (instead of a directory) so it passed the test despite failing to function.

You’ll find all this out if you turbo-nitpick the software.

Yeah… I haven’t poked at the “agentic” harnesses much. I do not like the idea of being unable to see what the model is doing. As I think I said earlier, I already greatly dislike the big companies hiding the reasoning tokens so I can’t see what it’s “thinking”.

1 Like

I prefer manually developed software, but let’s be honest, every corporate dev at the very least is using AI to ship code or they’re about to be unemployed (this actually happened to a friend recently).

That said, AI-developed != AI. AI code isn’t necessarily more or less capable or secure than human written code.

1 Like

Word Homies

The term “vibe coding” has become so muddy that it’s almost useless, mostly because it collapses two completely different workflows into one bucket:

  1. Blind chat-box hammering: Copying and pasting errors into a chat box over and over until the red text goes away. It eventually “works,” but the author has zero idea what side effects, unhandled edge cases, or permission leaks were baked into the the code
  2. Harness-driven systems development: Working inside an IDE harness where the model acts as a high-speed drafting partner against your actual repo, but you are driving the architecture, inspecting the diffs, and auditing the blast radius.

There’s a lot of tribal pushback right now from folks whose entire identity is wrapped around what Ai is quickly replacing (replaced?). But for anyone who actually manages infrastructure, syntax has always been the cheap part—system architecture, state management, and blast radius are what actually matter. (Look at that Em—Dash…)

Echoing what @computer-janitor said about dependency surfaces: I’d trust a compact, AI-assisted script written against the standard library that I have personally traced and audited over a “battle-tested” GitHub dashboard dragging in 400 unvetted npm packages just to render three buttons and an API call. Traditional open-source isn’t automatically secure just because a human typed it. Stuff like supply chain poisoning and abandoned dependencies are very real attack vectors on any network.

The divide between safe AI-assisted code and absolute network liabilities comes down to operator discipline.

The greatest utility of an LLM isn’t that it writes the boilerplate—it’s that it exposes you to patterns, libraries, or system calls you might not have pulled off the top of your head. But that only works if you have the self-control to hit the brakes the second it outputs something unfamiliar. If the model introduces an unfamiliar flag, socket call, or data structure, I have to go research it to death before that code ever touches a compiler, let alone a service.

That’s where blind chat-prompting falls apart. If someone doesn’t understand the underlying stack or OS permissions, they won’t even realize the model just bound a debug server to 0.0.0.0 or bypassed TLS verification to “fix” a connection error. The code functions, but the attack surface is wide open.

To @f41n’s carpentry analogy: you don’t need to hold a degree in metallurgy to know how to frame a wall that won’t collapse under load. Most solid homelab sysadmins and self-hosters are polymaths—the classic “a jack of all trades is a master of none, but oftentimes better than a master of one.” You know enough about networking, container isolation, file systems, and authentication to recognize bad structural integrity when you see it.

I run my own AI-assisted tools on my servers and network daily without losing sleep. Not because I have blind faith in a probabilistic text generator, but because the trust isn’t in the model but rather the verification pipeline:

  • Architectural boundaries: The tool gets scoped down to least privilege (read-only tokens, dedicated service users, segmented VLANs/subnets, no raw root/docker sockets unless strictly isolated).
  • Tooling harness over chat: Building with an integrated IDE harness where linting, type-checking, and git diffs force you to see every single line changing across the project.
  • Continuous scrutiny: If you don’t know why a block of code works, it doesn’t get deployed until you do.

I do this because I’m genuinely interested in the code and function there of and I learn more by taking the time to do it. (and I’m one of ‘THOSE’ sickos that like learning)

If you treat the model as a tireless junior developer that drafts what you architect, and you hold it to strict code review, you can get maintainable tools tailored exactly to your homelab. If you treat it like an oracle and just press “accept all,” you’re eventually going to have a very bad afternoon at some point. But then again, just have Ai fix it… right?

4 Likes

If you also tell the AI to disable the firewall and configure the NAT rules for public access you are going to have a problem, but someone doing that could also have installed a networking service and create a test account with test as the password and have the exactly same problem without any AI being involved.

Most models will tell you loud and clear when you are asking it to do something dangerous/stupid, and if you decide to ignore all warnings it is probably not an AI problem.

1 Like