Artificial intelligence agent sent malware package to software developer, and he almost had it built into the project

While developing a software project, an engineer asked the artificial intelligence agent to recommend a software package to solve one of the problems. The AI ​​agent suggested a malware package disguised as a familiar library, but fortunately, the company’s security protocols did not allow it to be installed.

Photo credit: Fotis Fotopoulos / unsplash.com

Sergey Fitsak, managing director of consultant and software development company Softjourn, talks about this. While at work, an engineer asked an artificial intelligence agent to advise him on a software package needed to complete a common task. He recommends a package with a sensible name and a familiar library format.

Luckily, Softjourn has a policy that its employees actually follow: double-checking AI-recommended software solutions to make sure they’re legal. The developers took a quick look at the kit’s source code on GitHub and discovered that it had few downloads and was created a few days ago. This aroused suspicion.

Mr. Fizak said attackers noticed that artificial intelligence models would sometimes come up with package names that sounded reasonable but didn’t actually exist. Cybercriminals register projects on large websites under these names, hoping that unwary developers will install it first and then start checking it. If Softjourn didn’t have proper security protocols in place, engineers would have installed malware. It’s unclear what kind of threat he posed – he may have stolen data or caused other harm. Sergey Fitsak points out that the inspection only takes a few minutes, but it helps avoid unpleasant consequences for the entire project.