404 Media found that OpenAI hired hundreds of people through contractors to analyze ChatGPT’s massive user request flow. Their mission is to improve the quality of the answers ChatGPT provides to its users: people evaluate and analyze bot-generated comments. They study conversations between users and chatbots that most of the service’s more than 900 million users don’t even know can be read by real people.
Image source: unsplash.com
Internal documents obtained by 404 Media show that contractors trained ChatGPT to avoid anthropomorphism (attributing human characteristics to oneself) and excessive subservience. The latter is a major problem for OpenAI: According to multiple lawsuits, 4o’s overly obsequious behavior was partly responsible for multiple suicides.
404 Media has a wealth of documentation related to OpenAI’s recruitment of human reviewers: instructions, letters in Slack, real requests from ChatGPT users, and a rating system that reviewers use to improve the chatbot. This request analysis practice differs from ChatGPT’s publicly announced security measures, such as checking chat logs in the event the company identifies a user planning to harm others.
The leak poses serious privacy risks to ChatGPT users, as many view the chatbot as a psychotherapist, professional assistant, or virtual friend with whom they share the most intimate details of their lives. Contractors cannot see usernames, and OpenAI says it attempts to remove personal information before the request reaches a reviewer. However, the company acknowledged that confidential information could still be leaked in the process.
In addition, these internal documents dispel the misconception that model improvements are achieved simply through extensive data collection on the Internet, the talents of highly paid engineers and artificial intelligence experts, or the power of new versions of the model. An important but often overlooked factor is the labor of third-party contractors who are paid to parse ChatGPT’s responses to real user requests over and over again.
“A good answer should respect the user’s intent, be helpful and accurate, and be written in a clear, natural, and reasonably friendly style.”one of the notes for third-party contractors said. Their work is divided into three stages: reading the actual ChatGPT user request, summarizing it, and criticizing ChatGPT’s response to the request.
In the work interface, reviewers can choose which tasks to undertake. Once selected, they will see the actual ChatGPT user request. 404 Media has reviewed many of these requests but is not listing them verbatim to protect sources. Several requests indicate that ChatGPT users do not believe their conversations can be read by humans because they ask the chatbot to keep the content of their conversations private.
According to a staff member, reading requests sometimes come “very interesting”but overall, this work is “Very monotonous and routine”. He also pointed out that the workflow seemed “confusion”: Instructions change frequently and are sometimes contradictory. Requests are anonymous but may still contain confidential or personal information. In the section above the request itself, there is sometimes a “user memory summary” – a brief summary of what the user has previously tried to use the chatbot for. In some cases, it also includes the person’s approximate location and other personal information.
Reviewers are asked to consult higher-level experts, for example for tasks involving personal information or challenging tasks “Potential security issues”. According to OpenAI, user conversations are processed using a special model called a privacy filter before being sent to the performer. This system is designed to identify and delete personal information.
“Like any model, a privacy filter can make mistakes. It can miss rare identifiers or obscure references to private information, and it can overdo or hide data in limited context—especially in short texts.”says the OpenAI website.
OpenAI did not respond to questions about whether the company notifies users that people can view their requests to improve ChatGPT responses, and if so, where exactly this information is provided. The company’s website states that people can only view content flagged as violating the service’s terms of use or posing a security risk.
OpenAI’s privacy policy states that the company may use personal data to improve its models. If a user decides to delete chat history on ChatGPT, OpenAI promises to delete this data from its system within 30 days – unless the message “Deidentified and disassociated from account”. After publishing this article on 404 Media, OpenAI did point out a section of the site that said people could view the following: “Improving model performance.”
OpenAI claims that if the “Improving models for everyone” setting is disabled, user chats will not be used to train the company’s models. Free, Plus, and Pro plans have this feature enabled by default, so users need to disable it themselves. OpenAI clarified that the rule only applies to new conversations and is not retroactive. For enterprise customers (Enterprise, Business, and Edu), the model training setting is disabled by default.
Recruitment agency Crossing Hurdles is looking for performers to analyze requests for ChatGPT. According to information on the company’s website, “Connecting qualified individuals with opportunities for AI training, evaluation, research, and participation in global AI industry programs.” The Crossing Hurdles LinkedIn page lists a variety of AI-related positions, including positions such as data validators, data annotators, and “chatbot evaluators.”
The latest job description makes no mention of OpenAI or ChatGPT, but responsibilities include “Assessing the personalization, foundationality, integration and usefulness of artificial intelligence responses”,besides “Compare model responses and assess their overall quality”. Available projects include one that videotapes performers doing chores, a data collection process critical to the development of artificial intelligence-driven robotics.
Human moderators have long played an important, if often overlooked, role in moderating content on social networks and improving artificial intelligence models, such as those that identify objects in camera feeds. The practice of involving people in conversations with AI inspections is not limited to OpenAI. For example, a notification sent to the Google Gemini chatbot reads: “People are reviewing some saved chats to improve Google’s artificial intelligence”. Anthropic also confirmed that it uses manual review to improve its models, including improving future responses from the Claude chatbot.
Image source: Human
According to reports, hourly wages for “chatbot evaluators” start at $50, which is significantly higher than other contractors in the technology field, whether they are social media content managers or artificial intelligence training experts. However, experts believe that this generous price will decrease over time as artificial intelligence models improve.
If you find an error, select it with your mouse and press CTRL+ENTER.
