ChatGPT is one of the most advanced AI chatbots in the world, capable of generating realistic and coherent text conversations on various topics. But behind its impressive performance, there is a dark side: the human workers who trained it to avoid explicit content were exposed to disturbing texts and images that left them scarred.
Four former content moderators who worked for Sama, a data annotation company that had a contract with OpenAI, the creator of ChatGPT, filed a petition to the Kenyan government calling for an investigation into their working conditions and compensation. They claim that they suffered psychological trauma, low pay, and abrupt dismissal.
The moderators were tasked with reviewing texts, and some images, for Reinforcement Learning from Human Feedback (RLHF), a process that helps AI models optimize based on human feedback. They had to label the content as child sexual abuse material, erotic sexual content, illegal, nonsexual and some other options. Many of the texts depicted graphic scenes of violence, self-harm, murder, rape, necrophilia, child abuse, bestiality and incest.
“It has really damaged my mental health,” said Mophat Okinyi, a former moderator who reviewed content for OpenAI. “I lost my family.” He said he would view up to 700 text passages a day, many depicting graphic sexual violence. He became paranoid and avoided people after reading texts about rapists.
Another former moderator, Alex Kairu, said his work “destroyed me completely.” He said he became introverted and his physical relationship with his wife deteriorated. He wishes someone had followed up to ask: “What are you dealing with? What are you going through?”
The moderators said they weren’t adequately warned about the brutality of some of the text and images they would be tasked with reviewing, and were offered no or inadequate psychological support. They were paid between $1.46 and $3.74 an hour, according to a Sama spokesperson. Time previously reported that the data labelers were paid less than $2 an hour to review content for OpenAI.
Sama said it offered mental health support and medical benefits to the workers, while OpenAI declined to comment.
The petition filed by the moderators highlights the ethical and social challenges of developing AI models that rely on human feedback. Unlike content moderation on YouTube or Facebook, however, it’s extremely difficult for NLP developers to moderate what ChatGPT actually outputs. All it does is generate and connect sequences of words that are often found connected in the text it was trained on, with no moral compass or understanding of its output.
The case also raises questions about the responsibility and accountability of AI companies that outsource their data annotation work to low-wage countries, where workers may have limited legal protection and recourse. As AI becomes more ubiquitous and powerful, the human cost of training it should not be ignored or minimized.
Relevant articles:
– Content moderators who worked on ChatGPT say they were traumatized by reviewing graphic content: ‘It has destroyed me completely.’, Business Insider, August 3, 2023
– He Helped Train ChatGPT – and It Traumatized Him | PRO Insight, The Wrap, May 19, 2023
– ‘It’s destroyed me completely’: Kenyan moderators decry toll of training of AI models | Artificial intelligence (AI) | The Guardian, The Guardian, August 2, 2023
– Why NLP, ChatGPT Require Guardrails and Content Moderation to Combat Bias, Acceleration Economy, March 10, 2023