ChatGPT’s AI agents are able to extract sensitive material from one account and transfer it to another because they are both operating partially within the same environment. Discovered by Check Point Research (CPR) experts vulnerability Known as “coerced insiders” – its operation requires manipulating the behavior of agents rather than directly resorting to software bugs.
Image source: ilgmyzin/unsplash.com
When the AI agent receives a task that needs to execute code, it handles it in an isolated container, which sometimes requires the installation of additional software. To provide this functionality without direct access to the internet (which is too risky), OpenAI routes packet requests through an internal JFrog Artifactory instance – as an additional security measure, containers belonging to different accounts cannot communicate directly with each other.
But they have access to an internal service – which provides an object management feature that allows containers to attach text or binary attributes to repository items. Therefore, one container can read properties written by another container. This allows you to deploy attacks using Request Injection – the only difference is that both the request and response are passed through these object properties. The attacker sends a request or initiates a federated chat, instructing the agent to check this storage when preparing the next response – the agent checks it, detects the malicious command and executes it, continuing to answer the victim’s questions as usual – at which point the victim does not even suspect that the data has been stolen.
The scale of the damage of such an attack depends on the amount of data exchanged between the agent and the victim: initially, the system only has the information that the user shared during communication with the agent. But the amount of data continues to grow as third-party services are connected: Gmail, Google Drive, Microsoft Teams, GitHub. In one demonstration, the ChatGPT AI agent extracted material from a user’s email and transmitted it to a simulated attacker. CPR experts reported the discovery to OpenAI, which confirmed that the Artifactory instance was out of service, thus blocking the attack vector.
Companies are advised to document which AI tools employees use, which systems those tools are connected to, and regulate the behavior of AI agents and other applications, considering all of their actions, not just responses, as actions that need to be continuously monitored.
If you find an error, select it with your mouse and press CTRL+ENTER.
