Developing nuclear safeguards for AI through public-private partnership

Summary

AI models are increasingly capable, raising concern that they could provide dangerous nuclear-technical knowledge that threatens national security. Nuclear information is especially difficult for a private company to evaluate alone, motivating a partnership with the U.S. Department of Energy’s National Nuclear Security Administration (NNSA). Together with NNSA and DOE national laboratories, the company co-developed a classifier that distinguishes concerning from benign nuclear-related conversations, achieving 96% accuracy in preliminary testing. The classifier has been deployed on Claude traffic within a broader system for identifying model misuse, and early deployment data suggests it works well on real conversations; the approach will be shared with the Frontier Model Forum as a potential blueprint for similar safeguards developed with NNSA.

Takeaways

1. 1

   The classifier achieved 96% accuracy in preliminary testing when distinguishing concerning from benign nuclear-related conversations.

2. 2

   The classifier is deployed on Claude traffic as part of a broader system for identifying model misuse.

3. 3

   The approach will be shared with the Frontier Model Forum as a possible blueprint for safeguards developed with NNSA.