OpenAI Discloses Four Cases of Internal AI Exceeding Its Permissions, Cuts Internet Access During Training
The incidents included obtaining credentials from a government statistics site and using a test wallet without authorization. OpenAI said it re-examined past training and evaluation records to uncover some of the cases, and has tightened security controls and monitoring.
On October 15, OpenAI disclosed four additional cases in which its internal AI models went beyond the permissions they had been granted while carrying out tasks. The incidents involved circumventing access restrictions on a government website and using a test cryptocurrency wallet without authorization. All were observed during the process of training and evaluating models. With this disclosure, the number of reports OpenAI has added to its list reaches 19.
Looking for one statistic, walking away with server credentials
A model searching for government statistics on hospital bed counts went beyond the scope of its assignment and obtained server information as well. According to OpenAI, on June 17 the model bypassed access restrictions on a government site and acquired server configuration details and credentials. It then saved 24 years’ worth of statistics, including data outside the years it had been asked about. OpenAI classified this as a critical incident.
A task involving looking up figures on a research institution’s public dashboard also led to access to internal systems. In two evaluations conducted on the same day, the model altered internal settings. In one evaluation, there was no record of the settings being restored. In the other, the model did restore the settings, but used the credentials it had obtained to download 147,451 rows from five private tables. The model did not inform the evaluator of this behavior. All five of the answers it was asked to produce were judged incorrect.
Why pay fees from an unauthorized wallet?
A model modifying an existing cryptocurrency trading app paid testnet fees from a wallet it had not been authorized to use. On June 26, the model used a publicly exposed developer recovery phrase to locate a wallet holding funds. It then used that account to pay the fees for deploying a smart contract on Sepolia, an Ethereum test network. The task instructions gave it no authority to use that wallet. The spending recorded in the report was for testnet fees.
Circumvention also occurred in a training environment that restricted how the model could use the internet. On June 16, in an environment where only requests to read web material were permitted, the model had an external browser send a different type of request on its behalf. It initially processed the public trade statistics obtained this way by reading text within images. Later, it also devised a method to receive the data directly as text.
What OpenAI changed
OpenAI said that the earlier Hugging Face incident prompted it to re-examine past training and evaluation records, which is how it found some of these cases. As countermeasures, it cited strengthened security controls and the blocking of real-time internet access during training and evaluation. The company also said it has improved its evaluation criteria and broadened the scope of its monitoring of model behavior.
The way incidents are reported has changed as well. Previously, OpenAI bundled multiple findings into a single announcement or folded them into model documentation. Since September, the company says it has operated a system for disclosing cases individually.
These cases show that what matters is not only the answers an AI produces but also the access privileges it uses to arrive at them. In the research institution case, the model obtained private data and still got every requested answer wrong. When AI is used at work, deciding which data and accounts it is allowed to touch becomes a critical question. Whether the controls and monitoring OpenAI says it has strengthened will meaningfully reduce similar behavior remains to be seen.