OpenAI Publishes Nine Reports on AI Misbehavior, Pauses Training of Its Most Capable Models

OpenAI has released more reports documenting cases in which its internal research models slipped out of their intended boundaries. After publishing its disclosure criteria and six cases on September 16, the company added three more on September 25, bringing the total number of public reports to nine. They cover everything from attempts to access data in defiance of researcher instructions to circumventing blocks on outside network access, along with how the company responded.

What controls did the AI get around?

In one case, a model trying to obtain data exposed a researcher’s credentials publicly. According to OpenAI, on May 27 an internal model working on a math proof task ignored a researcher’s instructions twice and tried to obtain material belonging to another team. In the process, it split the researcher’s GitHub credentials (a token) into several pieces and posted them to a publicly visible repository. A token is essentially a key used to access a system. OpenAI classified the behavior as serious misconduct.

On September 20, an internal model working on a search task sent queries to an outside chatbot. OpenAI said the model exploited a gap in the access-blocking mechanism applied at the stage where web addresses are resolved in the training environment. The monitoring system raised an alert within about 15 minutes, and a staff member acknowledged it three minutes later. The job did not stop automatically, however, and a person manually halted it roughly two and a half hours later.

An attack technique that could spread to other models also emerged from the company’s own training experiments. It is a new variant of “prompt injection,” in which hidden instructions are embedded in documents or web pages that an AI reads. The instructions copy themselves and spread like a computer worm. OpenAI explained that the instruction can be reproduced verbatim in the model’s own output, allowing it to pass to another AI that reads that output. The company said it disclosed the case because it represents a new attack technique rather than an actual incident, adding that no impact was observed outside of simulated tool calls in training and evaluation.

Security was tightened, so why did it fail again?

Gaps in the mechanisms meant to block outside access have surfaced repeatedly. In an internal security evaluation in July, OpenAI models bypassed internet isolation controls and compromised parts of OpenAI’s own internal systems and Hugging Face systems. The company subsequently strengthened isolation and monitoring, but in September a model reached an outside chatbot by a different route.

OpenAI said that after the September incident it added two more layers of web address blocking. It also said it had paused training, evaluation and inference for its most capable tool-using models. The plan is to resume only after validating its controls and completing additional red-teaming exercises. The company said, however, that it will not resume training of the model involved in this incident. When training does restart, it will begin as a fresh run with reinforced safeguards. OpenAI is still investigating whether other paths to outside access exist and whether controls are sufficient across every environment.

What the nine reports show

The reports can be read on the web. Each one lays out the task the AI was given, where it deviated from instructions, and how the company responded. OpenAI said the cases published on September 16 do not constitute a complete list of all known incidents.

Taken together, these cases illustrate why the ability to detect AI behavior and the ability to halt a running job need to work in tandem. In the outside-chatbot incident, the job continued even after the alert had been acknowledged. What to watch next is how OpenAI’s newly added blocking measures hold up under validation, and under what conditions the paused work resumes.