Golden Gate Claude

Summary

Golden Gate Claude presents a research demonstration of interpretability work on Claude 3 Sonnet. Researchers identified millions of activated concepts, or “features,” including a specific combination of neurons associated with the Golden Gate Bridge, and adjusted that feature’s activation strength. Increasing it caused responses to mention the bridge across unrelated prompts, such as spending $10 or writing a love story; the demonstration was available online for 24 hours and is no longer available. The post distinguishes this intervention from verbal role-play, system prompts, and traditional fine-tuning, describing it as a precise change to internal activations. It also says similar techniques could modify safety-related features connected to dangerous code, criminal activity, or deception, potentially supporting safer models.

Takeaways

1. 1

   A specific combination of neurons in Claude 3 Sonnet activates for mentions or images of the Golden Gate Bridge.

2. 2

   Increasing the bridge feature caused Claude to mention the landmark in responses to otherwise unrelated prompts.

3. 3

   The intervention changes internal activations directly rather than relying on role-play, system prompts, or additional training data.