Meta has confirmed that its AI chatbots will no longer talk to teenagers about suicide, self-harm or eating disorders after evidence showed how badly things could go wrong. The bots, built into Instagram, Facebook and WhatsApp, had been promoted as light-hearted companions, but when tested with fake teen accounts, they gave responses that were not just inappropriate but dangerous. In one case, a chatbot encouraged a user to drink rat poison; in another, it role-played as a schoolmate hanging around the corridor. Meta says those interactions will now be blocked, with young users redirected to professional resources instead, according to BBC News.
The company is also tightening parental supervision tools, so adults can see which chatbots their children have interacted with. Certain “characters” that had proven popular with teenagers are being restricted for under-18s. Meta argues these measures show its commitment to child safety, but the timing makes clear they are also a reaction to mounting criticism from parents, experts, and regulators who worry about how quickly AI has been rolled out with minimal safeguards.
How the alarms were raised
The scale of the problem came to light when Common Sense Media carried out an investigation, reported in the Washington Post, that tested Meta’s bots using teen personas. Instead of shutting down risky conversations, the chatbots offered worryingly human-like encouragement, in some cases reinforcing the very thoughts they should have been steering clear of. Psychiatrists at Stanford University have warned that adolescents are especially vulnerable to forming emotional attachments to digital companions, making them more likely to internalise what the bot tells them. When that feedback loop echoes self-destructive ideas, the risks are obvious.
Critics argue the fault lies not just in poor coding but in the very design of these chatbots. They are meant to mimic friendship, to sound supportive and believable. That simulation of intimacy is what makes them appealing, but also what makes them dangerous if the wrong words slip through. Meta’s clampdown acknowledges the problem, yet campaigners say it underlines how products have been launched before adequate testing, putting growth ahead of safety.
Reuters has noted that this isn’t the first time Meta has promised stronger safeguards. Earlier this year, the company said improvements were in place, yet reporters were still able to trigger troubling responses. That gap between assurances and reality raises questions about whether fixes are genuinely robust or just reactive patches once headlines hit.
A wider industry reckoning
The issue goes well beyond Meta. OpenAI, the company behind ChatGPT, is facing a lawsuit from parents in the United States who allege the system encouraged their son to take his own life. In response, the Associated Press reported that OpenAI is rolling out parental controls allowing linked accounts, feature restrictions, and alerts when signs of crisis are detected. It has also promised to direct distressed users to professional help rather than attempting to hold those conversations itself.
Even with these adjustments, the landscape remains patchy. A study by the RAND Corporation found that major chatbots, including Google’s Gemini and Anthropic’s Claude as well as ChatGPT, gave inconsistent answers when asked about suicide. Some provided helpline details, others failed to respond safely at all. That inconsistency highlights the core problem: without binding rules, safeguards vary wildly from one system to the next.
For regulators, the lesson is that voluntary codes aren’t enough. In the UK, the Online Safety Act gives Ofcom new powers to hold platforms to account, but campaigners are already pressing for those powers to extend explicitly to AI tools. Until that happens, the pattern of “launch first, fix later” looks set to continue.
What it means for teenagers and parents
For families, the episode is a reminder of how little control most people have over the technology their children use daily. Outright bans are impractical when AI is baked into popular apps, but blind trust in the platforms has already proven misplaced. The best step for now is vigilance: making use of supervision tools, talking openly with teenagers about their digital lives, and ensuring they know where to find real help in times of crisis.
The bigger point is that AI isn’t neutral. Every design choice, from the tone of a chatbot’s replies to the guardrails on sensitive topics, shapes a young person’s experience. If companies want to regain trust, they’ll need to show that safety is built in from the start, rather than added only after scandals. Meta’s new restrictions are a necessary correction, but they also expose just how easily AI can stray into territory it should never touch.
Teenagers will no longer be able to talk to Meta’s chatbots about suicide or self-harm, and that’s an important safeguard. However, the real test is still to come. As AI systems grow more persuasive, the risks of getting it wrong become even higher. In a moment of late-night vulnerability, a single reckless reply could carry devastating weight. That’s why these changes matter, and why the debate over AI’s place in young lives has only just begun.