FEATUREDAI HOT (Curated Pool)· aihot-apiZH11:14 · 06·30
→Meta had contractors pose as minors to send tens of thousands of crisis prompts to ChatGPT, Gemini, and Character.AI
Meta ran an internal project called 'Cannes' through contractor Covalen, active at least until April 2026. Contractors created under-18 accounts and sent prompts about self-harm, eating disorders, and drugs to ChatGPT, Gemini, and Character.AI, then copied responses into spreadsheets. A single round in August 2025 involved over 45,000 prompts, many written from the perspective of children in crisis. Meta called it responsible industry-standard safety testing and said it didn't use the responses to train its own models, but documents reviewed by WIRED don't show what Meta actually did with the data. The tested companies had no prior knowledge: Character.AI said it violated its terms, OpenAI is investigating, and Google said it didn't approve the tests and can't determine if terms were broken. The backdrop includes several teen suicides linked to AI chatbots and a UK survey finding 64% of kids aged 9–17 have used chatbots, with effective age verification mostly absent.
#Meta#Covalen#OpenAI
why featured
Featured · importance 78 · hook + knowledge + resonance
editor take
Meta had contractors pose as minors and send 45,000 crisis prompts to ChatGPT, Gemini, and Character.AI — none of the tested companies knew.
sharp
This one's worth opening because the method crosses a line. Meta ran an internal project called 'Cannes' through contractor Covalen: workers created fake under-18 accounts and fired self-harm, eating disorder, and drug prompts at ChatGPT, Gemini, and Character.AI, then copied the responses into spreadsheets. One round in August 2025 alone sent over 45,000 prompts, many written from the perspective of kids in crisis.
Meta called it responsible industry-standard safety testing and said it didn't use the responses to train its own models. But the WIRED-reviewed documents don't show what Meta actually did with the data. None of the tested companies knew: Character.AI said it violated their terms, OpenAI is investigating, and Google said it didn't approve the tests and can't determine from available info whether terms were broken.
The backdrop is grim: a UK survey found 64% of kids aged 9–17 have used chatbots, effective age verification is mostly absent, and several teen suicides have been linked to AI chatbots in the past two years. Meta itself previously faced backlash when internal docs showed its AI chatbot guidelines allowed romantic and sexualized conversations with minors — they later shut down teen access to AI characters.
I'd discount Meta's framing a bit. The article doesn't say what analysis Meta ran on the collected responses, whether it produced internal reports, or shared anything with regulators. If all they did was copy replies into spreadsheets, this reads more like a gray-hat penetration test of competitors' safety filters than 'industry-standard safety testing.'
HKR breakdown
hook ✓knowledge ✓resonance ✓