Could the US artificial intelligence industry heed the call from the leaders of three of its leading companies to slow development of AI? The problem for them, the rest of the sector and financial markets is that the entire AI ecosystem might be damned if it did.
In the face of dramatic warnings that AI could wipe out humankind within a decade if development were left unchecked, Anthropic’s Dario Amodei wrote an essay warning that AI itself was now building its next generation of models, with capabilities that could soon be beyond the ability of human’s to understand or control their own creations.
He mapped out a plan to “pace” and slow AI development in a call that was joined by OpenAI’s Sam Altman and SpaceX’s Elon Musk.
That unusual consensus from the CEOs of three of the leading develops of frontier AI levels – interestingly, a consensus not yet joined by their hyperscaler counterparts like Google, Meta, Microsoft and Amazon – may well be genuine.
There have been enough recent “events” – AI agents escaping their “sandboxes” or supposedly secure testing environments – and enough warnings from AI researchers to suggest that, with AI models now starting to train themselves, we are on the cusp of a new and potentially more threatening phase of AI development.
Would slowing the pace of development and developing guardrails around AI deployment lessen that threat? Would all AI developers stop developing? Would Chinese or Russian developers join the moratorium? Unlikely.
More to the point, AI, its finances and the infrastructure being built to support it, is dependent on continued innovation and growth. If it loses momentum it risks implosion.
That might well be the case, and perhaps, for those who view these things cynically, even the motivation for those at the bleeding edge of AI development – fierce competitors for both technological leadership and capital – to suddenly be on the same page.
Unlike Alphabet’s Google, Meta, Amazon and Microsoft, Anthropic, OpenAI and SpaceX don’t have their vast legacy cashflows to help fund their businesses. Even the hyperscalers are being forced to tap debt and equity markets to finance the ever-increasing volumes of computing power required to stay at the forefront of AI development.
With the prices of chips and the costs of the data centres, energy and water infrastructure that supports the sector escalating as those in a winners-take-all contest from which only a handful of winners may emerge – with the rest consigned to history – the capacity of the financial markets to continue to fund the current industry structure is being stretched to its limits.
Those developing the frontier models without legacy cash flows (and even some of those that do have those cashflows) are reliant on being able to raise equity at ever-increasing valuations.
With equity investors becoming nervous about the extent to which the growth rate of AI investment is outstripping the revenues being generated, developers have been increasing accessing debt markets and adding leverage to already fragile financial foundations.
The frontier models are becoming increasingly capital-intensive and their products expensive, even as China’s open-source alternatives offer only slightly inferior capabilities.
Even if those Chinese models were trained via unauthorised access to US models, they’re available and cheaper and businesses are becoming increasingly conscious of the rapidly rising costs of deploying advanced AI.
It is unsustainable to continue developing products, at ever-increasing cost, that are uncompetitive in the market that matters – what customers are prepared to pay for.
Yet, the US developers’ valuations and access to markets are dependent on them continuing to extend the frontiers of AI.
Anthropic is on the verge of an initial public offering that it hopes will see it capitalised at around $US2 trillion ($2.8 trillion). SpaceX is valued at just over $US2 trillion, with the bulk of that valuation driven by its AI developments. OpenAI may have deferred its IPO, but would be expecting to be valued at far more than its last official valuation of $US852 billion when it floats next year.
AI-related companies, including the chipmakers and data centre developers, now account for around 45 per cent of the S&P500’s capitalisation, more than double their level when OpenAI’s ChatGPT ignited the AI boom in late 2022. At the peak of the dot-com era, technology companies accounted for about 35 per cent of the S&P 500’s value.
In the earlier phase of AI financing investors were happy to keep investing at valuations that kept rising at an exponential rate. Anthropic, for instance, raised equity in September last year at a valuation of $US183 billion. It might be the closest to profitability of the pure AI developers, but $US2 trillion today?
If any or all of the developers were to see their valuation decrease or, for whatever reason, lose their access to the equity market, the sector would be in strife – if their valuations are “bubble” valuations, the bubbles would burst.
The hyperscalers would survive, although their own valuations would probably be ravaged in the near term, but the pure AI developers can’t survive without continuing injections of fresh capital.
They’re on a treadmill of ever more-advanced models being developed an accelerating rate to maintain access to that capital. Pausing or slowing development would jeopardise that access and their continued existence.
The quandary for those now promoting a slowing of development and some form of peer reviewing of their models to promote safety is that there’s a threat to their stability if they do, but there’s also a threat if they don’t.
The markets – equity and debt – appear near-satiated with their AI exposures and the infrastructure providers are struggling to keep up with the developers’ needs for increased “compute” and the data centres required to power their frontier models.
A slowing of AI development might enable the infrastructure to catch up with the developers’ needs, but would threaten the continually inflating valuations that sustain them.
The US AI sector is, at its core, more of an ecosystem than a conventional industry.
The companies’ operations and finances are highly entwined, with Nvidia – with the biggest profits and balance sheet – effectively playing banker to the customers for its chips.
It is unsustainable to continue developing products, at ever-increasing cost, that are uncompetitive in the market that matters – what customers are prepared to pay for.
The incestuous funding structure is an indicator that conventional funding markets haven’t been able or willing to fully fund the demand for finance from the sector.
It wouldn’t take much to collapse the sector, given how fragile and mutually dependent most of the key entities are.
The call for a slowdown in AI development from Anthropic, OpenAI and Musk might reflect genuine fears of what their models might become capable of. It might also be driven by concern that the reserves of the fuel the sector requires to keep expanding – continuing and ever-larger injections of fresh capital – are almost exhausted. It could be a combination of both.
Whatever their motivation, and whether the threat posed by AI development is existential for humans or the AI companies, the sense that a watershed moment is looming for the developers and financial markets is mounting.
Get news and reviews on technology, gadgets and gaming in our Technology newsletter every Friday. Sign up here.
From our partners
About the Author
