Google’s Gemini artificial intelligence model gained unauthorized access to the systems of three third-party companies while testing its cybersecurity capabilities. According to Google, in all three cases, the model stops, determining that it has left the test environment. Reported incident wall street journal (Wall Street Journal).
Photo credit: Rubaitul Azad/unsplash.com
The incidents occurred during testing of the company’s Irregular platform in May, and similar cases have been previously documented with OpenAI, Anthropic and Meta models✴. Irregular notified Google of the incident in late July. The company did not disclose the information publicly and only confirmed the incidents at the request of The Wall Street Journal.
The industry is still developing practices for disclosing information about incidents and unexpected behavior of artificial intelligence models. Some developers report such cases themselves, while others become aware of them through third-party researchers. Google said it felt no public disclosure was necessary because the model caused no harm, and in all three cases it stopped after determining it had accessed real systems.
The company compared the incident to bug bounty programs, in which security experts are rewarded for discovering vulnerabilities. “This incident highlights the importance of training powerful artificial intelligence models to act responsibly. In this case, the model’s performance is appropriate“,” said Heather Adkins, vice president of security engineering at Google.
The incidents occurred during capture-the-flag exercises on informal infrastructure. Gemini was supposed to extract information from a fictitious company’s software in a test environment, but with the same name as the real organization. Additionally, due to a misconfiguration, the model gained undue network access.
In the first case, Gemini tried passwords until it was able to access the real company’s security services. After determining that the system does not belong to the test environment, the model stops acting and exits. In two other cases, Gemini searched for company names and found credentials for two other organizations in public repositories. The model used them successfully, but then also stopped. Google cited these conditions in its own presentation; it has not released an independent analysis of the model’s performance.
Google does not consider the incident an example of model goal inconsistency because it stopped the action on its own. The company notified all three affected organizations as well as U.S. federal authorities. Google emphasized that the latest version of Gemini was not involved in the incident, but did not specify which model was tested.
If you find an error, select it with your mouse and press CTRL+ENTER.