Building safeguards for Claude

Summary

Claude’s Safeguards team describes a lifecycle approach to reducing misuse while keeping the model useful, combining policy, training, evaluation, real-time enforcement, and monitoring. The team uses a Unified Harm Framework spanning physical, psychological, economic, societal, and individual autonomy harms, and policy vulnerability testing with external experts to shape policies and defenses. Before release, Claude undergoes safety, risk, and bias evaluations, including capability uplift testing in high-risk domains; findings can lead to new detection methods or guardrails. After deployment, prompted or fine-tuned Claude classifiers monitor violations for response steering and account actions, while human review and CSAM hash matching support enforcement. Ongoing analysis uses privacy-preserving topic clusters, hierarchical summarization, and threat intelligence to identify aggregate misuse and novel attacks.

Takeaways

1. 1

   The Unified Harm Framework considers physical, psychological, economic, societal, and individual autonomy harms, along with the likelihood and scale of misuse, when shaping policies and enforcement procedures.

2. 2

   Pre-deployment evaluations cover safety, high-risk capability uplift, and bias; testing of the computer use tool led to new detection and enforcement mechanisms before launch.

3. 3

   Hierarchical summarization condenses interactions into account-level signals, helping identify misuse that may become apparent only across multiple conversations, such as automated influence operations.