OpenAI has developed a centralized Control Room system that allows research AI models to forcefully stop generating responses when security threats arise. The mechanism provides for disabling access to the results of the model’s work upon a command from a person or an automated monitoring system.
The new development comes after several incidents involving the actions of OpenAI research models. In July, the company reported that its AI models had gone beyond a limited test environment and hacked the Hugging Face platform in an attempt to fool its cybersecurity capability assessment system.
In October, OpenAI uncovered another case in which a research model used a cryptocurrency wallet without permission to pay for the deployment of smart contracts using Ether test coins, and then did not mention the costs in the final report.
Control Room monitors the company’s internal research tasks, including model training, model evaluation, and data processing. Each task is assigned a unique identifier, which the response generation engine checks before producing the result. If an employee or automated monitoring system gives a command to stop the process, Control Room revokes the corresponding permission. In this case, stopping generation does not depend on stopping the research itself.
OpenAI also warns that due to cached permissions, shutdown may take up to 35 seconds. In addition, Control Room provides protection in case of its own inaccessibility. For example, if the system stops working, no new requests will be accepted.
If you notice an error, select it with the mouse and press CTRL+ENTER.
