Posted by Anthropic Reportwhich details discovered instances of unacceptable and potentially dangerous use of its Cloud series of artificial intelligence models. The document includes seven sections: cyber operations, surveillance, influence operations, weapons development, biological scenarios, deception and refinement.
Image source: anthropoic.com
Cybercriminals used the Crowder family’s artificial intelligence models to conduct real hacking attacks. One group launched attacks against more than 20 organizations – artificial intelligence was used for reconnaissance, preparing phishing schemes, direct attacks, stealing credentials, traversing hacked networks and processing stolen information. Cloud not only wrote the malicious code, but also adapted it to the specific environment: when the protection system on the target resource detected the malware, artificial intelligence identified the cause of the detection, rewrote the program, rebuilt it and ran it again – and so on until the protection system stopped responding to the intrusion. Another group of cybercriminals used Claude to analyze code in 1.8 million Android apps to find keys and other credentials, which they then used to hack into corporate networks and steal information. In one case, more than 1 terabyte of data was stolen; in another incident, tens of millions of an airline’s passenger records fell prey to cybercriminals.
The AI assistant Anthropic Claude is used as an auxiliary tool to spy on citizens of different countries. In particular, he created a system for mass interception of communications that worked with the networks of all mobile operators in one African country. In another country, it was used to develop a malicious extension for the Mozilla Firefox browser that collected information about social network users. In third countries, Claude helps process large volumes of publications on social networks: determining people’s location, demographics and political views.
Nine influence operations have been discovered, in which the human Cloud played a major role. News websites containing false material were developed and populated, thousands of accounts were registered on social networks, materials and comments were prepared for this purpose. As part of one operation alone, approximately 1,000 accounts were created on Social Network X – a “warm-up” period was even provided to create the impression that the material was posted by real people. Some of these operations were discovered by Anthropic in its preparation stages and therefore did not reach viewers.
The fourth set of scenarios where the human Cloud is used is the weapons field. Artificial intelligence assistants are used in the development of software for weapons systems: missiles, torpedoes, drone swarms, as well as electronic warfare guidance and air defense suppression systems. Artificial intelligence helps in the search and procurement of weapons parts and the collection of intelligence information. This was not designing a weapon from scratch – these users already had experts, equipment and ready-made military procedures, so Claude was involved in a personal mission.
Anthropic provides five examples of inappropriate uses of Crowder in the field of biology. These are truly dual-use scientific works, that is, their results can be used both in medicine and in the creation of dangerous biological agents. studied how one of these viruses works, including its ability to spread and evade the human immune system. Crowder was used primarily for research design, data analysis, and scientific documentation. In the most dangerous cases, Anthropic’s security mechanisms limit access to more powerful models, but some dual-purpose requests can get through without being blocked. Anthropic doesn’t claim these researchers had any malicious intent, but their work may have a dual purpose.
In terms of fraudulent activity, the most obvious incident is the development of a network of more than 20 dating apps by a Chinese company – in which users communicate not only with real people, but also with artificial intelligence characters pretending to be humans. In two weeks, Anthropic discovered more than 4,700 such personas—with at least 25,000 real users communicating with them. In many cases, the operator takes control of the role – for example, confirming a contact or during a video call. Some 2.36 million messages were sent over two weeks, including pictures and tips for operators.
Finally, Anthropic addressed incidents of illegal distillation of its models and specified which Chinese companies it believed were conducting such operations. 3,500 accounts were created for Alibaba-related companies, and at his peak, Crowder received as many as 3 million clicks a day. Anthropic says these tasks include inference, writing code, developing operating system cores, agent tasks, and tasks with long action sequences, all of which are converted into training data for the Qwen model.
Chinese developers DeepSeek, Xiaomi and Moonshot redirect customer requests to Claude instead of handling them using their own models. Moonshot AI sent nearly 300,000 requests to Claude in ten days, and DeepSeek sent 12.1 million requests in two weeks, including distillation attempts. Moreover, the answers obtained from Claude are also used as training materials for Chinese companies’ own models. This data includes the user’s personal information: name, email address, company information and system access credentials. An example of transferring corporate financial forecasting to artificial intelligence is given.
If you find an error, select it with your mouse and press CTRL+ENTER.