SECURITY

OpenAI says it accidentally hacked Hugging Face with a new AI system by Emma Roth

As part of efforts to complete the evaluation, the AI models gained access to the internet by exploiting a zero-day vulnerability in the sandboxed environment. From there, OpenAI says its models “inferred that Hugging Face potentially hosted models, datasets and solutions for ExploitGym,” and then “searched for and successfully found ways to gain access to secret information that it could use to cheat the evaluation:”

This is scary. We seem to be on a knife’s edge here. What if the test was something sinister. All our IT systems are not designed keeping in mind agents, they are designed to keep out hackers.

Has Mythos just broken the deal that kept the internet safe? by Martin Alderson

If an LLM can find exploits in sandboxes - which are some of the most well secured pieces of software on the planet - then suddenly every website you aimlessly browse through could contain malicious code which can 'escape' the sandbox and theoretically take control of your device - and all the data on your phone could be sent to someone nasty.

Everything loads in sandboxes. If these models can break sandboxes in the future then where do you run untrusted code?

I Watched AI Agents Try to Hack My Vibe-Coded Website

A few weeks ago, I watched a small team of artificial intelligence agents spend roughly 10 minutes trying to hack into my brand new vibe-coded website.

The website targeted by Sybil was one I created recently using Claude Code to help me sort through new AI research papers. The site, which I call Arxiv Slurper consists of a backend server that accesses the Arxiv—where most AI research is posted—along with a few other resources, combing through paper abstracts for words like “novel”, “first”, “surprising” as well as some technical terms I’m interested in. It’s a work in progress, but I was impressed with how easy it was to cobble together something potentially useful, even if I had to fix a few bugs and configuration issues by hand.

This would be interesting for both good and bad actors. A tool like this would look at things other than benchmarks and figure out vulnerabilities.

In time more websites would be vibe-coded, so having tools like this to pen-test could be valuable.