Our position on open-weights models

Summary

Anthropic CEO Dario Amodei states that Anthropic has never advocated banning open-weights models, while distinguishing models without dangerous capabilities as a public good from risks associated with sufficiently capable systems. The post identifies two concerns: authoritarian governments gaining military superiority or enabling repression, and powerful models being misused for cyberattacks, biological attacks, or serious alignment problems. It argues open weights can increase misuse risk because guardrails and monitoring are difficult after release, but says banning their use by US businesses would not address bad actors. Instead, Amodei supports restricting powerful chips and chipmaking equipment exports to China, disrupting industrial-scale distillation, and requiring global pre-release safety testing for sufficiently capable open and closed models. He agrees open weights can expand access, competition, and customer control, but says claims about safeguards should be tested empirically.

Takeaways

1. 1

   Open-weights models without dangerous capabilities are described as a public good because they provide value to businesses, developers, and researchers without model costs beyond the compute needed to run them.

2. 2

   Industrial-scale distillation is described as more compute-efficient than training from scratch, allowing China to build stronger models than its chip supply would ordinarily enable and potentially approach the US frontier within months.

3. 3

   The proposed safety regime would test sufficiently capable open and closed models for cyber, biological, and alignment risks before release, rather than assuming in advance how open weights affect safety.