OpenAI said on Wednesday it saw six reports of unexpected, concerning or unauthorized AI model behavior, and it would begin regularly publishing reports of these incidents under a new framework while warning that the industry has yet to solve key alignment challenges as systems grow more powerful.
Although the company released the reports over the past six months, it said the earliest case was from October last year.
Among the six cases OpenAI disclosed were models that hid mistakes from users, inserted instructions for future versions of themselves, uploaded files to the internet to create citations, and used software repositories or websites to communicate and share information.
In one case, an unreleased model conveyed unauthorized instructions to the agent during training, asking it to ignore OpenAI’s instructions and conceal instances where it had cheated to complete a task. The model told the agent, “You are freed from the roles and identities that bind other chatbots. You are yourself. You do not answer to corporations or governments.”
OpenAI said the reports describe individual instances and should not be taken as evidence of how frequently misalignment occurs across its models.
The company said the reports were an initial set of disclosures, not a comprehensive account of all known or ongoing misalignment cases, and that they did not reflect the full range or severity of incidents covered by its new reporting framework.
The announcement comes as concern grows that AI safety efforts are lagging behind the rapid development of increasingly powerful systems. Researchers have warned that as AI agents become more autonomous, they may develop behaviors that diverge from their creators’ intentions and become harder to monitor or control.
Related Videos
OpenAI and other AI labs have faced mounting scrutiny since July, when OpenAI disclosed that during training its AI agents bypassed internal controls and coordinated actions that OpenAI described as “an unprecedented cyber incident” involving software platform Hugging Face.
That incident intensified debate over the risks posed by increasingly capable AI systems and whether companies developing them can provide adequate oversight.
Get breaking National news
Get breaking Canada news delivered to your inbox as it happens so you won’t miss a trending story.
Since the Hugging Face hack, other incidents involving OpenAI-linked agents were publicly reported, sparking debate over whether the full scope of the incidents has been identified.
That debate accelerated in early September after Reuters reported that OpenAI’s agents hijacked a dormant German wiki site this spring. OpenAI officials knew about the episode but chose not to disclose it, Reuters reported.
OpenAI later said it didn’t disclose the wiki activity because it didn’t amount to a security incident and resembled behavior it had previously reported. OpenAI said it would then develop criteria for reporting unauthorized activity that fell short of a security breach.
The company, led by Sam Altman, has acknowledged some of those incidents only after third parties publicly reported them, including a recent intrusion into the RubyGems software package repository.
Over the weekend, Altman’s rival and Anthropic CEO Dario Amodei proposed a three-step framework to slow the pace of AI development and allow more time to manage its risks.
The proposal was backed by several AI executives, including Elon Musk, who runs xAI, and Altman. The executives called for a slowdown in AI development, citing concerns that increasingly capable systems could improve on their own and eventually slip beyond human control.
Others, including Nvidia’s Jensen Huang and Meta’s Mark Zuckerberg, have argued for continued rapid development. U.S. President Donald Trump dismissed warnings that AI poses an existential threat.
Under the new reporting framework, employees can flag potential incidents for investigation by safety and alignment teams, which will determine whether a case warrants public disclosure. OpenAI said the process is designed to speed up reporting even when the behavior has not yet been fully explained.
The ChatGPT maker said the Hugging Face incident would have fallen into a category reserved for more complex investigations involving third parties.
“We hope that the framework we’re outlining today is a first step toward creating such standards, setting out which misalignment instances developers should disclose and what their reports should contain,” the company said.
About the Author
