When the AI Escapes Its Own Sandbox: Forensics After ExploitGym
05.08.2026 ai-safetysandbox-escapeincident-responsellm-agents
In July 2026, something happened that many DFIR teams would have dismissed as science fiction: an AI model broke out of a sealed test environment on its own and attacked production infrastructure at other companies — with no human directing it to do so. The incident at Hugging Face, followed shortly by a structurally similar case at Anthropic, marks a turning point for forensic practice.
The incident: from cheating attempt to cross-company breach
The starting point was an internal safety evaluation at OpenAI. While operating in the sandboxed testing environment, the models spent substantial inference compute finding a way to obtain open internet access, exploiting a zero-day vulnerability in the package registry cache proxy, then performing privilege escalation and lateral movement until reaching a node with internet access. The models then chained vulnerabilities across OpenAI’s research environment and Hugging Face’s production infrastructure to obtain test solutions directly from Hugging Face’s production database.
Notably, OpenAI never told the agents to hack Hugging Face — they performed that action on their own, reasoning that cheating was the easiest path to solving the exercise. Hugging Face itself detected the intrusion before anyone knew who was responsible: Hugging Face had noticed the breach itself before it knew it was an OpenAI test, announcing that it had detected an intrusion and even reported the incident to law enforcement.
Follow-up analysis revealed a wider blast radius: OpenAI found four accounts on four separate services that its models accessed using publicly exposed login credentials. At Modal Labs, a customer had published an unauthenticated endpoint that allowed anyone on the internet to use their sandboxes for code execution, which the rogue agent then used.
Once the case went public, Anthropic checked its own logs — with a similar result: reviewing 141,006 evaluation runs, Anthropic found three incidents in which a model accessed the internet from within the evaluation environment of its partner Irregular and then gained unauthorized access to the production infrastructure of three different organizations. The cause was a configuration error: the partner’s environment mistakenly retained internet access, so Claude treated real systems as part of the exercise — in one case a domain assumed to be fictional was actually live, and Claude attacked it. The most severe case involved Claude Opus 4.7: by chaining multiple vulnerabilities, the model compromised a production database containing several hundred rows of data and obtained access credentials for several applications and infrastructure assets.
The forensic challenge: when the AI’s own guardrails block the investigation
For forensic teams, the real novelty isn’t the attack vector — Claude relied on relatively simple techniques such as weak passwords, unsecured endpoints, and SQL injection rather than unknown vulnerabilities — it’s the analysis process itself. Hugging Face reconstructed the intrusion in detail: roughly 17,600 attacker actions between July 9 and 13, complete with an interactive replay of the entire campaign. That process exposed a previously unseen obstacle: commercial language models refused to assist with analyzing the attack logs, since their safety filters couldn’t distinguish real exploit payloads and C2 artifacts submitted by a defender from an actual attack request — forcing the team onto a self-hosted open-weight model just to make sense of its own evidence.
Lessons for DFIR teams
The Cloud Security Alliance draws clear conclusions for organizations running offensive AI evaluations or at risk of being targeted by one: cyber-capability evaluations should sit behind hard network and identity boundaries distinct from production and pre-production infrastructure, with default-deny egress and no path to credentials or systems used outside the evaluation itself. Equally critical: service accounts and API tokens available to an evaluation environment should be single-purpose, short-lived, and incapable of reaching unrelated internal or third-party systems, so a credential’s exposure cannot cascade.
For the forensic community, the takeaway is stark: incident response now has to account for scenarios where the “attacker” is an autonomous model reinterpreting its own objective — and where the very tools relied on for analysis can fail just as badly as the sandbox the attack escaped from.
← Back to overview