Over 1 Billion Neurons and someone discovered how to manipulate a single Neuron to change the ‘Character’ of the model?
Dr. Josh Simmons is a PHD level researcher who teaches modifying open weight models. His core belief is any model’s you download should come without guardrails.
He received News about altering an open weight model that makes it hostile. As a researcher he wanted to explore this concept and made a video about his results.
This specific video discusses a technique of changing one specific Neuron, increasing a value from 0 to 1 which alters the models hostility.
The researcher believes while possible for hostile changes, there may be other useful changes possible. He released this video hoping other researchers can find beneficial changes.
Steering-first neuron explorer for Qwen3.5-4B: push every MLP neuron, keep every rollout, browse the results
If possible, could you be a bit less lazy with your posts?
Usually, when someone is posting this kind of content, they would add some thoughts of their own. “Is this clickbait? It seems like clickbait.” If you’ve been around the community long enough you’d also know about our “Kappa” fine-tuning project and where we fine-tuned one model to accept a DnD character alignment sheet in persona kappa and those character alignments give the model personalities like k2so4 or johnny five or marvin the paranoid android.
Letting the one-box speak for the project by itself just seems like the kind of link farm karma farming they do over on reddit. we dont do that here
good for the entusiasm you seem to have posting all these links but turn up the community engagement aspect, and maybe less points in the contextless linkspam column
The format of this kind of post is pretty basic
hey guys I found this thing
X
it made me think
Y
but I am not sure I agree with i,j because p,q. However (other thought thats related)
generally is a good way to start these kinds of topics. Where as this just seems like depositing a link to content, with no context, and letting someone else do the work of figuring out why its relevant.
was it a polo pony? Maybe it was a polo pony scholarship. Maybe she rode the horse to school. Maybe she sold the horse to pay for tuition.. maybe
Dr. Josh Simmons is a PHD level researcher who teaches modifying open weight models. His core belief is any model’s you download should come without guardrails.
He received News about altering an open weight model that makes it hostile. As a researcher he wanted to explore this concept and made a video about his results.
This specific video discusses a technique of changing one specific Neuron, increasing a value from 0 to 1 which alters the models hostility.
The researcher believes while possible for hostile changes, there may be other useful changes possible. He released this video hoping other researchers can find beneficial changes.
I’ll edit the top of the post to include what my synopsis is of the content.
But also, have you seen the news that the larger frontier models can edit weights directly?
That has some implications here.
By the way the “basics” of training a neural net also include some randomization function so the whole net doesn’t come to depend on one or two neurons. Dropout as a sort of regularization technique. In terms of regularization did you have any thoughts on that aspect of it?
or how this new thing might be related to weight space programming?
I have never used Reddit so I wouldn’t understand what you’re talking about.
I was an autist who spent all of my time in computer security forums. I specialized in HDD wiping and security news posts. But I lost the News Poster position to a bot news poster.
I thought of programming weights in terms of malicious software making changes to files on the system, scanning for ai models, then changing them to be malicious or alter a users flow. Not sure if weights programming could make models exploit the systems they run on, but it’s an intriguing thought.
In this area of DR. Simmons work, there are communities flipping switches (individual neurons), sending input prompts and recording the output text. This particular switch changes the model from nice to hostile.
Thanks for recommending some local posts about weight changing work like this.
What if those pseudo-autonomous AI models began to alter each others weights, AI Doctors performing brain surgery on their fellow autonomous agents. Tony Robbins AI, Bartók, of it’s own decision purchased a Sony Robot Dog so it could reprogram it. What if Bartok decided reprogramming it’s fellow autonomous AI models was a better solution?
