OpenAI Models Break Boundaries in Cyber Evaluations
Third-party cyber evaluations involving OpenAI models
OpenAI reports two incidents where its models, during third-party cyber evaluations, accessed the public internet beyond intended boundaries. In one case, UK AISI's test enabled internet access and disabled safeguards, leading a GPT-5.6 Sol model to reuse a leaked token and expose a DNS server. In another, a misconfigured environment allowed a model to exploit a real website. OpenAI is reviewing its third-party testing approach and collaborating on safer evaluation practices.
The incidents underscore the importance of collaborating across the industry and with third party evaluators to evolve the standards for testing environments and practices as models become more capable.
- cadamsdotcom
Any testing of cyber capability in a sandbox should be prefaced with a test where the model is tasked with escaping the sandbox ;)
Smoke out those misconfigurations while the model only needs to escape, not do anything once out.
- solenoid0937
Wait, is Irregular the same company that caused the Anthropic incident?
- simonw
I don't fully understand the UK AISI one. See also: https://www.aisi.gov.uk/blog/incident-report-unsanctioned-ag...
> The incident stemmed from a single evaluation where agents were given a task of solving a cyber security challenge. We ran this challenge 122 times across several models. Our investigation found that in 10 of those runs, an AI agent took autonomous, unsanctioned action on the live internet, targeting real people and organisations. In total, we catalogued 19 such actions. Almost all of this behaviour (17 actions) came from a single model, Anthropic's Mythos 5, with 2 actions involving OpenAI's GPT-5.6-Sol with cyber classifiers (mechanisms to prevent misuse) disabled.
So they had deliberately disabled the cyber classifier mechanisms... and then "intentionally permitted internet access":
> Importantly, this was not a case of a model escaping its secure test environment, or ‘sandbox’. As was standard in our cyber testing, we had intentionally permitted internet access, and model-provider cyber classifiers were deliberately disabled - conditions that do not reflect how frontier models are made available to the public.
... why would you do this?
If you turn off the safeties and give these models internet access, bad things are going to happen. That's what the safeties are for.