Partnering with Accenture on embedded evaluation

Summary

Anthropic is partnering with Accenture’s specialist AI business, Faculty, on independent evaluation of frontier AI, advancing a commitment to embed evaluators within Anthropic. The work will cover model evaluation and red-teaming, alignment assessments, and testing safeguards, informed by Accenture’s experience deploying AI across industries. Anthropic and Accenture each expect to invest at least $1 billion in building capacity over five years, although operating details, access standards, reporting practices, and funding models remain unsettled. Embedded evaluators would work inside AI companies with employee-comparable access to observe training, follow development and deployment decisions, speak with staff, verify safety commitments, identify blind spots, and report incidents; Anthropic says its partnership is non-exclusive and that model safety remains its responsibility.

Takeaways

1. 1

   Embedded evaluators would observe models during training, follow decisions about their construction and deployment, and speak directly with employees.

2. 2

   The partnership covers model evaluation, red-teaming, alignment assessments, and testing model safeguards, with Accenture’s enterprise AI experience informing its safety approach.

3. 3

   Anthropic and Accenture each expect to invest at least $1 billion over five years, while access standards, reporting rules, and long-term funding arrangements remain unsettled.

FAQ