Artificial intelligence agents are gradually entering the life of modern man. They relieve users of everyday tasks, but their mistakes threaten to have unpredictable consequences. AI agents can publish confidential information in a work chat or make purchases even without access to payment instruments.
Image source: Mohamed Nohassi / unsplash.com
A user of the AI agent Grok Bot, Shane Mac, described how he posted his bank statement in a group chat on the corporate messenger Slack: checking and savings account balances, data on expenses for building a house and regular payments for Netflix and car insurance. For personal purposes, he used an AI agent, which he named “Personal Financial Director”, to which he gave read-only access to his bank accounts. Another AI agent on the same platform had access to the work chat in Slack – he was called “Chief of Staff.”
Mr. Mak demanded an explanation as to how it happened that the “Personal Financial Director” published confidential information in the work chat. The AI agent explained the situation and even apologized in his own way: “My fault. Personal finance is not a suitable topic for this audience, period.”.
AI agents have exploded in popularity in recent weeks, being trusted with managing correspondence, planning trips, and a host of other tasks. But at some point they begin to behave unpredictably, just like the models that control them, which are prone to hallucinations, misinterpretation of data, and even spontaneous actions on the verge of cybercrimes. Unlike chatbots, which only answer questions, AI agents can perform specific actions on behalf of the user, for example, putting things in order in the calendar or making an appointment.
The agent breaks the received command into a number of stages, which, for convenience, are not announced to the user. They ask permission before taking important actions, such as purchasing goods or deleting files, but otherwise act at their own discretion. If the user’s request is unclear, or there are multiple ways to perform the required action, confusion can occur. In Shane Mac’s case, the AI agent confused group chats with similar names. The Grok Bot developers assured him that the problem had been fixed.
Annica Benning, who works in San Francisco, has been using the AI agent Instinct for almost a month now, and you can interact with it directly through the messaging app on your iPhone. She uses the app for online shopping, ordering Uber taxis and booking airline tickets. In 90% of cases the system works flawlessly, but there are some surprises.
Image source: Igor Omilaev / unsplash.com
One day, a woman returned a swimsuit to an online store and asked the AI agent to suggest replacement options. He sent her a link to confirm her choice, but she, according to her, never confirmed the purchase. However, I received an order confirmation email. When performing transactions with a bank card, the system sends a link for confirmation, but in this case the payment was made using funds from the refund, so the system was able to bypass the confirmation stage. It was not possible to cancel the order – now Ms. Benning hopes to return it when it arrives.
In another case, while traveling to another city, she instructed Instinct to book a table for dinner – he booked, but not for that evening, and on this evening she had other plans that he knew about. The woman had to ask the agent to cancel the reservation and negotiate with the restaurant administration so that she would not be charged a $50 fine for no-show—the restaurant, through the agent, agreed to accommodate.
Shalini Dinesh from Austin preferred the Anthropic Claude chatbot, but decided to try Meta’s new AI agent✴ Muse. The woman entrusted him with organizing a party in honor of her son’s 13th birthday. The party was taking place at a paintball court, so the AI agent composed and sent messages to parents with a consent form to sign; the message also specified what clothes the children should wear. The system found nearby restaurants ready to deliver food, taking into account allergies of some participants; and selected themed gift sets for children according to age. But she couldn’t cope with the situation when it turned out that two children had the same names – the AI agent counted them as one person, but Shalini Dinesh managed to double-check the list and correct the error.
Jesse Levey, a Bay Area startup founder, tried using Meta✴ Muse to manage the schedules of my three children. He began to manually transfer data about the children’s standard weekly schedule from his calendar to Muse, but realized that this would be too troublesome, given that sections and clubs use specific applications, and joint trips are discussed in group chats.
Children are engaged in the ski section, and it has a special application with a calendar; There are also SMS, WhatsApp, email, calendar applications and about 65 other applications that the system would need access to. Then Mr. Levy asked Anthropic Claude if there was a more powerful AI agent that could manage his children’s schedules, taking into account the requirements for privacy and security. He offered to hire a person for this – he will definitely be able to “chasing for places in summer camps, improvising when children are sick, and making a dozen decisions a week that no system can handle”. Creating an AI-based version of such a system “would require several months of your attention”Claude warned.
If you notice an error, select it with the mouse and press CTRL+ENTER.
