A debate over whether artificial intelligence could have something resembling consciousness has put Anthropic, one of the world’s leading AI companies, at odds with the Vatican after Pope Leo XIV rejected the idea that machines could replace or surpass human beings.The disagreement surfaced after Anthropic co-founder Christopher Olah reviewed the Pope’s first encyclical, “Magnifica Humanitas”, ahead of its launch at the Vatican. Olah had accepted an invitation to speak at the event on May 25, but reportedly considered pulling Anthropic out after disagreeing with the Pope’s position on AI consciousness.The dispute points to a much bigger question surrounding the rapid development of AI. As companies build increasingly capable systems and deploy them in everyday life, there is still disagreement over whether these systems should be understood simply as tools, autonomous agents or something closer to beings.
What did Pope Leo XIV say about AI
In his encyclical, Leo warned against allowing technology to become more important than human beings.” … humanity — in all its grandeur and woundedness — must never be replaced or surpassed”He also wrote, “If technology becomes the ultimate criterion, the human person risks being reduced to data, a cog in a machine or a commodity.”The Pope’s position clashes with questions Olah has been raising privately about whether advanced AI systems could possess some form of moral status.According to The New York Times, Olah has encouraged religious thinkers to consider the possibility that Anthropic’s Claude models could have something resembling moral status.
Why is Anthropic thinking about AI’s moral status
Anthropic has spent months consulting religious scholars and other experts about how AI systems should behave and what values they should follow. The company has held private discussions with scholars from around the world and required participants to sign nondisclosure agreements, according to the Times.The effort comes as Anthropic pursues rapid expansion, with the company potentially reaching a valuation of $2 trillion.Olah has argued that AI development cannot be separated from questions about values and morality.“Those who control AI will impose their own moral vision, which will become the invisible infrastructure of these systems,” Olah argued, per the Times.His concerns go beyond philosophical questions. Anthropic has been examining possible real-world risks from AI, including disruption to jobs, cyberattacks and biological threats.AI systems are already being introduced into workplaces, customer services, search, education and other areas where their decisions can affect people’s lives.
Anthropic’s attempt to decide what AI should value
Anthropic says it is trying to address these concerns through outside consultation and a written “constitution” for Claude.The 84-page document is intended to guide how the AI model behaves. The company has also consulted philosophers, religious scholars, psychologists and civil society representatives.Anthropic’s in-house philosopher Amanda Askell said she wants the company’s models to be “the best of us”. Olah, meanwhile, has argued for “some shared notion of goodness that cuts across society in some very broad way,” according to the Times.But questions remain over who should ultimately decide what values powerful AI systems should follow. Private discussions between technology companies and outside experts may influence AI behaviour, but they do not amount to public oversight.Olah has also said researchers are encountering behaviour in AI systems that they do not fully understand.“I will be honest: We keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment,” Olah said, according to the Times.At the centre of the dispute is therefore a question that goes beyond the Vatican and Anthropic. As AI becomes more capable and increasingly embedded in society, researchers, technology companies and religious thinkers are still debating what these systems actually are, what rights or moral status they might have, and who should decide the values that guide them.
