Any examples of AI improving a project's code

Has there been any notable examples of the as of right now very high level coding “skills” of AI being put into good use by improving a project’s code in novel ways? Any open source project would count, third party kernel drivers, productivity suites, OS level improvements… I’m thinking from a consumer perspective

Just to add my own opinion:
Yeah, I think there is a notable improvement in the security of software, while paralell construction occurs in the direct opposite direction.

1 Like

When straight coding where the AI generates the whole thing it typically turns out really badly. But the latest thing has been using it to find bugs in software, which AI can apparently do VERY well. More major bugs have been found across major software platforms this year than any other in history due to AI use. Though the numbers could be slightly skewed, since AI was used the last couple years to generate some of the code on many of those platforms.

1 Like

Who said it has high level coding skills? Their own marketing team?

As it is now, its more like a somewhat highly autistic junior dev (autism is the closest human analog to the fact its a machine, it will not lose focus or get bored, its always “on” no matter how soul crushing the task might be). It can scan giant codebases and find mistakes nobody noticed for a decade, or help write documentation, or do the first stage of code review for code submissions to a project.
But if you ask it to write a lot of complex code it will faceplant very quickly and produce hacky garbage.

Openwrt has implemented an ai code review bot (donated tokens from Anthropic afaik), and its pretty good at noticing issues or code that does not follow the project standards, and suggest ways to improve it.

1 Like

A sad, very sad day for intelligence agencies worldwide. I remember some of the first AI detected cves in the kernel were in cryptographic modules that were a great place to hide a couple privilege execution vulnerabilities.

3 Likes

Nah, chances are very high they have a backdoor or insider access to most if not all operating systems. If they don’t have access the operating system vendor may just hand over the info gained through “telemetry” (looking at you $micro$oft also this) .

1 Like

Yeah, and those “vulnerabilities” are their backdoors for Linux desktop/server. One thing is hiding vulns in places nobody cares to spot them for years and another is trying to get a full blown backdoor like they (allegedly, in my opinion) have on Windows/Mac/whatever past the kernel maintainers (and anybody else watching). They had to be a bit more stealthy.

Now they will have to use AI to make “vulnerabilities” that are less detectable by AI, or plan for a much faster vuln obsolescence cycle.

That’s also kind of not a thing on linux desktop/server

It’s kind of unclear what it means to improve code in this context.

Since AI is (probably) excellent at inline documentation and refactoring, beautification, etc. it could be argued that even inexpensive or relatively outdated models can “improve” an existing, functional codebase. These are areas where even great developers typically struggle (nobody likes writing documentation, etc.).

But I suspect you are asking more about AI impoving the code by adding new features, fixing bugs, and general automation of the development process. This is a much more subjective question, in my opinion. Does an AI produce more bugs than the average human developer? Well, again, this is somewhat subjective. Does it make sense to buy tokens as a FAANG company that can attract top programming talent? I think that’s still a much more questionable proposition, even given the state of frontier models.

And of course none of these considerations reach the c-suite or the stock market floors where people have become totally detached from the business realities.

The xz backdoor would not have been found if it were not for a sysadmin who spotted odd behavior , a regression and excessive cpu usage by ssh. This was a long game by a potentially state actor as the hacker contributed to xz for years before introducing the backdoor.
Also ever hear of salt typhoon? You know the backdoor mandated by law in the US for all the telecom companies so the US govt can spy on phone calls and texts ? Yes that still exists…

Ubuntu has some telemetry, its appears to be opt-in though. Whos to say that Ubuntu, Redhat, or whoever has that backdoor built-in knowingly or not?
A backdoor doesnt need to be an error in coding that introduces a vulnerability to be exploited, it could be perfectly good code that does a job and potentially provides a backdoor as a secondary function. Thus it wont be found by AI scanning for bad code/vulnerabilities.

I’m really not sure that I personally even know of a top level software company that isn’t deeply using things like Claude, cursor, etc as a large part of their development at this point. So whether or not that software is “better” remains to be seen it’s at least turning out code faster that is passing all of their unit tests and being shipped out the other side.

As far as “highly autistic” since when is that not a good trait for a software developer?

Somewhat related story

Pointing out that the consumer is faced with shitty updates, with blame on LLMs

I was able to vectorize a bit-matrix transpose using AI that I was not able to do myself. It encoded the transpose matrix in a random-looking unsigned long then used the 512-bit Galois field instruction to perform 8x8 byte sections of the bit-matrix.

is a good example of what I said. That’s a fully fledged backdoor and it lasted only between versions 5.6.0 and 5.6.1 of xz tools before being discovered. In real world time that’s a few months, from February to March 2024. Also it was discovered before this version could reach the LTS distro releases (its intended target most likely, also because it was targeting the builds of deb and rpm packages for x86-64). It took around 3-4 years of effort for the hacker/agent playing “good boy” and using sockpuppets to get commit privileges to the repo and this thing in btw.
Also no AI code reviews were involved at the time, this was done by humans testing the software and noticing strange behaviours.
It was a resounding failure. The complexity of a true backdoor is its main weakness.

Closed source stuff

also closed source stuff

Taking the xz backdoor as an example, I’m strongly suspecting an AI code review would notice that one of your distro’s downstream patches is starting a rube goldberg maneuver where the build system is pulling down a random guy’s tar.gz, extracting scripts that extract other scripts and replacing a function in a library without clear intent of doing this in the commit message. This behavior is very far from “normal safe development practices” it was trained on. OpenWrt’s current code review bot can catch every single instance of submissions doing stuff that is not clearly explained in the commit message for example.

I’m not saying you can’t fool a modern AI bot review, just that it’s going to be more involved than fooling a (probably overworked as is often the case in opensource) human reviewer

I started using it for security reviews and testing after i heard about project glasswing. just general codex with a security review skill rocked my world.

I’m solo-founding a new company and the core product is something i had hacked together over years in python. I worked with codex to refine the ‘true’ architecture i was after, then re-design the entire repository structure for multi-dev, extension, in rust. It’s doing the re-implementation now. I have to baby it a lot. It wants to cheat so bad. It will create interfaces and mock services to pass tests and forget about if if you let it go too long horizon. I’ve definitely opted for slicing the work up almost as if I was writing everything myself and then taking a quick gander at files, review, and descriptions. Even doing small slices, it drifts down rabbit holes i’m glad I catch and reset the session (it tried to build and android emulator from scratch instead of using xcode. freakin’ try hard.)

1 Like

This is a terrible assumption. Maybe the most competent use of the best models can produce good results here, I dunno, but in my experience so far, everything people do with language models is considerably worse than everything they used to do by hand. It’s an excuse to engage in lazy wish fulfillment and avoid due diligence.

I don’t know if I should name names, but personal experience, there was a big monied company that did payment processing that very obviously adopted LLMs to rewrite all their API documentation, say in the past year timeframe. I found a bunch of narrowly described technical documents with completely missing and/or miscategorized navigation and context. The lack of human intentionality and organization was striking. The details weren’t characterized well enough to be usable. They were just dumped out of something somewhere.

I was myself personally the unhappy recipient of that slop, and wasted several weeks trying to get support from that company before they contradicted their own documentation saying that a feature we were going to rely on (because their docs said to) didn’t exist yet. That’s why the not-so-enlightening errors were happening. After they finally got back to us, after egregiously wasting our time asking for barely-on-topic additional information repeatedly, they completely deleted all of their docs and replaced all of them.

A second case, a different, smaller vendor, they’ve clearly been using LLMs for their documentation, and it’s all overly verbose now with off-topic textbook stuff mixed in with the relevant information. It also has these weird asides where it congratulates itself in weird language for using “industry standards” like metric units. For some reason they gave their “new contributor” a human name.

FWIW, KDE likes to phone home when I open the menu, and Firefox likes to phone home when I open its menu. Bottles (friendly GUI wrapped around management of Wine prefixes) phones home a lot. This big tech norm of constantly monitoring you is now well represented in the open source space.

I would suggest that…anywhere CodeRabbit has been properly implemented is probably a good candidate for a codebase that’s been materially improved.

In fact, I’d say that if I was going to keep just one AI dev tool and throw the rest away, it’d be CodeRabbit. It’s just excellent at what it does, and - critically - doesn’t muck around with your code. It just alerts you to potential issues and asks you to straighten up and fly right.

Not open-source but recently at work I did a huge refactor that combined several separate but related repositories we had which duplicated quite a bit of functionality, and even some code. Claude code was a huge help in extending the test suite, doing the mechanical changes (my typing spend is abysmal compared to an LLM), and identifying and diagnosing bugs. Merged into the main branch after 4 weeks, and it’s a huge relief to me. The result is a much cleaner, better-tested codebase that I still know back-to-front. I could have done all of this without AI tools, but it would have taken much longer (2 - 3x) - a no-go in small startup trying to find PMF.

While my experience was positive, I think a lot of the discourse about AI lacks nuance. You have to know where its strengths and weaknesses are, and use the right tool for the job. It’s fantastic for large mechanical refactors and I think it can be really useful if you take a pair-programming approach (even if it’s just acting as a reviewer), but it’s a shitty architect and you need to make sure you aren’t being lazy and offloading key design decisions and mission-critical work to it. As a developer, you are responsible for what gets committed and deployed. I think a lot of the exuberant vibe-coders moving fast now are going to be in for a shitty time in the coming months when slop they pumped out without a careful review starts breaking in prod. That said, I think experienced devs who use it appropriately can produce the same or better quality output in less time (maybe 0 - 50% more productive, depending on what the actual work is). The thread is specifically about quality, but there are other use-cases where quality is less important: one off scripts, internal tools, etc. where an LLM can basically one-shot it without much supervision. That’s cool too.

Sorry my post above was quite off topic. To contribute something that’s actually on-theme I think this interview (from last year) includes interesting examples about where AI was used to improve open-source projects:

Both the interviewer (Richard Feldman) and interviewee (Mitchell Hashimoto) know what they’re doing. I use both Zed and Ghostty and they are both high-quality tools IMO!