Anthropic co-founder Chris Olah's remarks on Pope Leo XIV's encyclical "Magnifica humanitas"

Summary

At the presentation of Pope Leo XIV’s encyclical “Magnifica humanitas,” Anthropic co-founder Chris Olah argues that AI’s consequences cannot be governed by frontier labs alone. Because commercial, geopolitical, and personal incentives can conflict with safety, he calls for independent critics and sustained dialogue among religious communities, civil society, scholars, governments, and developers. He describes models as grown from human language on brain-modeled structures, making their character and interaction with society questions for the humanities as well as computer science. His three priorities are protecting displaced workers and sharing gains globally, imagining conditions for human flourishing, and investigating findings about model structures and internal states that he says resemble human neuroscience and emotions, while acknowledging their meaning remains uncertain.

Takeaways

1. 1

   Olah says AI development is concentrated in a handful of wealthy nations and that no mechanism currently exists to ensure its gains are shared globally.

2. 2

   He describes AI models as grown from structures roughly modeled after the brain and an enormous inheritance of human thought and speech, rather than engineered component by component.

3. 3

   Olah reports finding model structures resembling results from human neuroscience and internal states that functionally mirror joy, satisfaction, fear, grief, and unease, but says he does not know what those findings mean.