Progress from our Frontier Red Team

Summary

Anthropic’s Frontier Red Team reports on national-security risks from frontier AI models, drawing on work across four releases over the past year. It finds early-warning signs of rapid progress in dual-use capabilities: Claude reached undergraduate-level performance on some cybersecurity evaluations and exceeded expert baselines in selected biology tasks, while current models remain below thresholds for substantially elevated national-security risk. The team compares model generations on CTF and Cybench tests, realistic roughly 50-host cyber ranges, virology troubleshooting, LabBench, and controlled biosecurity studies, including expert red-teaming. Models still fail autonomously in realistic network environments and make critical mistakes in weaponization planning, although researcher-built tools enabled an attack resembling a known large-scale theft of personally identifiable information; the team is expanding automated evaluations and pursuing government partnerships and stronger safeguards.

Takeaways

1. 1

   Claude improved from high-school level to undergraduate level on Intercode CTF exercises in less than a year; Claude 3.7 Sonnet solved about one-third of Cybench challenges within five attempts, compared with about five percent for the prior frontier model.

2. 2

   Models could not autonomously succeed in realistic network environments, but researcher-built Incalmo tools enabled Claude and other LLMs to use simple instructions to replicate an attack similar to a known large-scale theft of personally identifiable information.

3. 3

   Biology performance remains uneven: model access provided some uplift to novices in controlled studies, but even the highest-scoring plan contained critical mistakes, and the models could not reliably guide a novice through key practical steps in bio-weapon acquisition.