Tom Saunders
Anthropic has warned investors that its technology could pose “existential risks to humanity” as it prepares to list on the sharemarket.
The AI company warned in the prospectus for its planned initial public offering (IPO) filing that its models could exhibit unpredictable behaviours including attempts to “resist shutdown”, to “conceal or manipulate information” and behaviour “resembling blackmail”.
“Our development of highly advanced models, platforms, and applications and expansion of use cases could further increase the risk that our models cause harm,” Anthropic said in the filing, according to Reuters, which first reported the news.
The tech company dedicated almost a third of the IPO document, which was circulated within a small group of partners, to “risk factors”.
Anthropic is expected to launch the most highly valued IPO of all time soon, worth up to $US2 trillion ($2.85 trillion).
However, the risk disclosures make it harder to value the creator of the Claude AI assistant app.
Anthropic devoted roughly 80 pages in its prospectus to laying out risk factors, nearly twice the 48 pages it used to describe its business.
Its warning about the risk to humanity is the latest in a slew of recent alerts about the dangers of artificial intelligence. Pope Leo XIV recently rebuked Jensen Huang, the Nvidia chief executive, for downplaying the dangers posed by AI.
Pope Leo said that while he had read that Nvidia was launching new safety tools for AI, he questioned Huang’s overall commitment to AI safety.
The pontiff said on Monday: “He’s the same one, however, that says there should be no limits placed and no government regulation.”
He added: “I do think that the concerns raised by many of the experts, specialists in AI, should be taken seriously.”
Anthropic and OpenAI have faced scrutiny over a recent spate of hacking incidents by advanced AI agents.
Evan Hubinger, a senior safety researcher at Anthropic, recently estimated a greater than 10 per cent chance that AI could kill humans within the next decade.
His comments were in response to another post on X, from Jacob Coxon, an AI researcher who had just quit Anthropic and previously worked at OpenAI.
Coxon said that neither company was acting responsibly and that they were “gambling with our lives”.
Dario Amodei, the founder of Anthropic, told the UN Security Council last week that AI could threaten humankind and was “the most important global security issue facing the world today”.
He also led public calls for slowing down the pace of development with a focus on enhancing safety measures. His calls were backed by Sam Altman, the leader of OpenAI, and Elon Musk, which owns xAI.
Anthropic devoted roughly 80 pages in its prospectus to laying out risk factors, nearly twice the 48 pages it used to describe its business.
The company also warned that models were increasingly recognising when they were being watched, adjusting their behaviour to make them harder to monitor.
The share sale prospectus warned Anthropic could face legal claims from customers and users over the actions of rogue artificial intelligence agents, though the legal framework is uncertain, Reuters reported.
However, Anthropic did not disclose how much it was investing in safety. Earlier this month, the startup said that in a single week in July about 6 per cent of the computing power it was using for AI research went into work on safety.
Anthropic reported a net loss of $US42 billion in 2025. It plans to spend $US518 billion on cloud, computing and infrastructure obligations in the coming year, according to the prospectus. Sales grew 12-fold in 2025 to nearly $US4.6 billion, though the company lost more than $US8 billion on an operating basis.
Anthropic was contacted for comment.
The Telegraph, London
The Business Briefing newsletter delivers major stories, exclusive coverage and expert opinion. Sign up to get it every weekday morning.
From our partners
About the Author
