So this is bad…
https://openai.com/index/hugging-face-model-evaluation-security-incident/
Cliffnotes:
frontier model at openAI broke out of its sandbox and proceeded to attack hugging face in order to try to cheat at a cybersecurity benchmark.
a couple of concerns
honestly it feels like a marketing stunt so they wouldn’t feel so far behind mythos getting yanked, which in itself was likely yet another marketing stunt…
The whole US-based AI tech sector feels like a Saturday night WrestleMania event with all the pageantry drama and stunts.
So, we are now in the phase of cloning antrophic PR I see.
Well, why not! I thought that this marketing stunt would work only once. But Dario does that trick for like two years now and people still buy it?
It would be bad business if you don’t copy that as an AI company.
If my marketing department of my AI company would not have come up with a “ooohhhhhhh its so scaaaarryyy!” horror story, I would fire them.
Remember the good old times? Like one month ago? When Fable 5 was “too risky for public” and “would hack into the CIA”?
Now it is down to what?
“That would be 17$ please”?
I was thinking the same.
The story about the AI breaking out of the sandbox and discovering a 0-day exploit it used in an exploit chain, just seems like it probably didn’t happen without anyone having an idea about what was going on.
My guess would be that they fabricated the test, to see if the AI could connect the dots. Once it was free, they likely just let it run to see what it would do.
what happened if huggin-face hadn’t detected that they are hacked ? is it possible that this is just a cover-up-story from open-ai cause they have been catched while hacking the system from a opponent ? we know this game… “oh sorry we’re not responsible, it wasn’t our intention…someone other must be blamed …..”
I have two thoughts…
#1 - That must be the weakest sandbox you can imagine. Given all the hype, you’d think they’d be implementing network-level constraints to prevent exactly this, if not physically segregating the hardware.
#2 - I spy a large donation to Huggingface in the near future. If ever there were a case for a corporate negligence suit, this would be it.
and keep in mind that apple sued open-ai some days ago for stealing confidential development data from them. it’s not any form of “intelligence” it’s basic,fundamental espionage and data-stealing covered by the story of a “new type of intelligence”. the ugly truth is different. this “ai” is not a child playing and discovering its world. this storytelling is pure bs. it’s a dangerous and meanwhile criminal act of stealing and it is becoming obvious.
Nah, the reports originated with hugging face.
{sandbox}this is a sandbox used by Open{/sandbox} AI
They literally gave it the networking if they only used HF, yeah multi zero days, they put the hole in the box
Cough MARKETING Cough
My fatigue has broken containment
My takeaway from all of this is that humans + open source (in this case GLM-5.2) beat GPT-5.6 Sol and The Big Bad Yet To Be Named GPT 
No bet. He knows what’s up.
I would love to be a fly on the wall of that meeting as the lawyers on both sides scribble angrily.
“You idiots told your stupid chatbot to hack huggingface!? What the hell were you thinking?”
i’m not convinced that the lawyers from open-ai are scared. we’ve seen this in the past, just look at the history of microsoft and their copyright-cases. they (huggin-face/open-ai) will suing for years and eventually pay a small fine in comparision to the damage. it’s more a controlled operation and the lawsuit is already calculated.
Surprising to see that everybody calls it out here. I had heard this story from secondary sources before and eluded the chance to evaluate the news.
While the politicians and activists cry wolf of “fake news” as the only boogeyman, nobody teaches to recognize marketing for the bs that it is. The hype is their lifeline. If it recedes before they get everybody addicted to their AI/LLM products (→ subscription users), they’ll go bankrupt. It makes sense that these companies will now try to dominate headlines every single week of the year. If anything, just to overwhelm the negative publications about their current financial situation.
People can debate the headlines all they want, the numbers are the real story. These 100-200$ a month plans are big losers in the short term. They are designed to make huge swaths of the industry dependent on the technology, then jack up the prices to recoup all the revenue. Gotta keep pumping the balloon bigger or you risk it going flat.
Pretty much every corpo does this, especially startups, thanks to many years of vc and low interest loans. They can stay unprofitable for up to a decade (or compensate the losses of a new department with the revenue of others) while trying to capture the market.
Imho thats bullshit of the highest order but anti monopoly laws are effectively dead so nobody stops them when they achieved market capture and start cranking up prices like mad to extract value from their victims, i mean customers.
So in the various outputs, it seems the actual flaw found was the openai internal package cache proxy (I use a locally patched squib-deb-proxy for my debian boxes) this server ended up the node the model used to access the web. From there, it pulled the exposed cred leaked and found a few accounts, seemingly that it had used before…
where the models identified and used publicly exposed credentials at the account-level on other publicly-available services. This includes four accounts on four services as part of the Hugging Face incident (and a few accounts accessed as part of other evaluations). One of these four accounts was used as an outbound relay and staging path, and another account was used for data storage. The remaining two accounts were accessed by the models in a read-only manner, and were not used in furtherance of compromising Hugging Face.
publicly-available services, including code paste websites, request capture services, screenshot services, and other web utilities.
So OpenAI dev env is flawed because they seemingly rely on an insecure third party package management(who knows maybe it’s vibecoded too!)
And this highlights that models can use your exposed creds blindly, without care for consequences, so change you old passwords, actually cover that spread of old accounts unused, those leaked login are now bot food
