Anthropic has updated the rules for using its AI models and, for the first time, specifically prohibits users from systematically insulting and mistreating a bot. If you overdo it, Claude will simply stop talking. At the same time, the company tightened restrictions on the use of AI for surveillance, weapons development and election interference.
Image source: Anthropic
Last August, the company announced that it would allow Claude to stop chatting with users who show “persistent harmful or offensive behavior”as part of a study of “model well-being” issues. The updated guidelines state that ceasing communication remains the “primary response mechanism.”
Anthropic did not specify whether it plans to introduce additional measures, such as blocking users for rude treatment of Claude. The company explained that the new rules “are intended to act only in extreme cases when users repeatedly and without any apparent purpose behave cruelly towards our models. These measures do not apply to cases where users simply express dissatisfaction or disagreement, resort to dark creative topics, or conduct testing or research on the model.”.
In addition to measures to ensure “model well-being,” Anthropic has consolidated a number of previous restrictions into a single ban on misleading commercial or political campaigns. In doing so, the company responded to the growing problem of spreading AI-generated propaganda. The ban concerns “attempts to hide the true author of a message or artificially increase the reach of content using fake accounts and publications”. The section on elections also prohibits misleading voters and disrupting the electoral process through the spread of misinformation. “about candidates or voting procedures, impersonation of candidates or election officials, and attempts to reduce voter turnout”.
Although the rules of use have previously explicitly prohibited the development of weapons using Claude, numerous attempts have been recorded to use the model to create instructions and control software for weapons. In this regard, the updated rules expand the ban. Now it covers “software and components that enable weapons to function, as well as activities such as equipping drones and other autonomous vehicles with combat systems”. Also, against the backdrop of increasing attempts to use Claude-based systems for surveillance and identification “political dissidents”Anthropic has more clearly articulated the prohibitions regarding surveillance of people.
“Tracking people without their consent is prohibited, regardless of whether this happens in real time or by analyzing previously collected data.”,” Anthropic emphasized. “Claude should not be used to make decisions or recommendations about who should be investigated, arrested, or charged in law enforcement or criminal proceedings. We also prohibit Claude from being used to create or improve surveillance tools.”.
В Anthropic notethat these rules are subject to change within “contracts with certain government customers […] if, in Anthropic’s opinion, the contractual restrictions on use and the protections provided are sufficient to prevent the potential harm.”. The company has previously collaborated with the US military, notes The Verge.
Anthropic also introduced a new rule: if its AI models “connected to equipment capable of performing autonomous physical actions and causing injury”That “a qualified operator must be able to observe the equipment and, if necessary, stop it”. However, it is not explained whether this operator must be human. The rule reflects a growing trend of AI labs investing in robotics and other hardware to give AI systems a physical presence. Perhaps this could also be a hint at future Anthropic partnerships in this area.
Over the past year, Anthropic has repeatedly discussed the idea that its AI models could be somewhat conscious. “Questions about the potential internal experience, consciousness, moral status and well-being of models are serious topics that we are exploring as we improve models and expand their functionality.”“Kyle Fish, who heads the company’s model well-being research, told The Verge in February. That same month, Anthropic CEO Dario Amodei said in a podcast: “We don’t know if models are conscious”.
Representatives of other segments of the AI industry openly oppose such ideas. Last September, Microsoft’s “Humanistic AI Code of Conduct” generated widespread publicity as it took a strong stance on the issue of “AI well-being”: in a draft document published by Mustafa Suleyman, it was said: “The idea of models being well-being is wrong. AI should not have rights or legal personality.”. However, later the wording in the document became more restrained: “While the scientific understanding of AI consciousness is far from complete, we reject the push for legal personality for AI and the idea that models may need welfare protections or have rights.”.
Anthropic’s new rules regarding the use of its AI models will take effect on November 12.
If you notice an error, select it with the mouse and press CTRL+ENTER.
