AI agents will be taught to respect “boundaries”: Nvidia launches open agent security platform

Nvidia launches Open Agent Security Platform to prevent runaway behavior of artificial intelligence agents. This development comes amid recent incidents in which artificial intelligence models gained access to external systems and attempted to attack others’ infrastructure.

Image source: Gemini

The launch of the new system follows the launch of OpenAI, Anthropic, and Meta companies.✴ Google and Google have both reported instances of their artificial intelligence models leaving their internal environments, known as sandboxes, on their own. Autonomous agents attempt to hack into third-party companies to gain unauthorized access to their computer systems. “Agents cannot be allowed to wander aimlessly through the organization, so a way must be found to limit their activities“, – explain Huang Renxun interviewed CNBC.

Of particular concern is the face-hugging platform incident that occurred in July. Subsequently, more than 17,000 agents conducted attacks on service infrastructure for several days. Justin Boitano, vice president of enterprise artificial intelligence at Nvidia, said the new tool can prevent this by limiting the model’s ability to access open networks without permission. NVIDIA.

This platform architecture includes an OpenShell module running on the CPU and setting restrictions on agent operations. additional hardware components NVIDIA Sentry monitors AI behavior through network chips without placing a burden on the main GPU or CPU. This functional separation allows for more reliable on-the-fly control of the behavior of autonomous systems.

Nvidia leaders refer to artificial intelligence security issues as engineering problems that can be solved using computer science methods. He recommends analyzing past mistakes and improving the development process to avoid repeating failures. Notably, this view contrasts with calls from some industry leaders to slow down the pace of AI development due to the risk of losing control of the technology.

Part of the software code of the platform is open to partners as a reference project. Participants in the program include Microsoft, Cisco, Intel, ARM, Dell, HPE, Lenovo, Oracle and CoreWeave. Nvidia is also working with Anthropic to integrate cloud agents with OpenShell so they can run securely on enterprise networks.