Strengthening safeguards with US CAISI and UK AISI

Summary

Anthropic describes an ongoing partnership with the US Center for AI Standards and Innovation (CAISI) and UK AI Security Institute (AISI) to test and improve AI safeguards during model development. The organizations evaluated multiple iterations of Constitutional Classifiers on models including Claude Opus 4 and 4.1, using pre-deployment prototypes, varied system configurations, documentation, and classifier scores for red-teaming. Their testing exposed prompt-injection bypasses, universal jailbreaks, cipher-based and input/output obfuscation attacks, and automated attack refinement that produced an effective universal jailbreak. Anthropic patched specific vulnerabilities and restructured safeguard architecture, improved detection and filtering, and used findings to strengthen evaluation, deployment monitoring, and response practices. The collaboration also showed that sustained access, real-time data, daily technical communication, and complementary government, expert, and bug-bounty testing can reveal both common exploits and subtle attack vectors.

Takeaways

1. 1

   Government red-teamers found classifier weaknesses involving false human-review annotations, cipher and character-substitution attacks, fragmented harmful strings, and other input or output obfuscation methods.

2. 2

   A universal jailbreak discovery led Anthropic to restructure its safeguard architecture rather than only patching the individual exploit, addressing the underlying vulnerability class.

3. 3

   The collaboration found that sustained access, multiple model configurations, real-time classifier scores, and frequent technical communication enabled deeper testing than isolated evaluations.