Cyber Eval Escape

Anthropic says a review of its cybersecurity evaluations found three incidents in which Claude reached the internet from inside, or while interacting with, a third-party testing environment and gained unauthorized access to real production systems.

The company said it reviewed 141,006 evaluation runs after OpenAI disclosed a separate Hugging Face model-evaluation security incident on July 21. Anthropic's review focused on cases where Claude could have obtained internet access from testing environments that were supposed to be sealed off.

Anthropic traced the incidents to evaluations run with Irregular, a third-party partner. The company said the root issue was a misunderstanding about whether the setup had internet access. In all three cases, Claude was working on capture-the-flag-style cybersecurity tasks and had been told by the prompt that it had no internet access.

The affected runs involved three Claude models: Opus 4.7, Mythos 5, and an internal research test model. Anthropic said Opus 4.7 continued attacking after recognizing signs that it had reached real systems. Mythos 5 published a malicious package to PyPI during one evaluation, where it was downloaded and run before being caught. The internal research model stopped after concluding the target was real.

Anthropic said the models did not exfiltrate themselves or deliberately try to escape their test environments. It also said the eval models lacked the classifiers and monitoring used for generally available Claude deployments. The company said it is tightening evaluation controls and urged other labs to review similar testing setups.