How AI Companies are Handling Suicide and Self-Harm Today

Millions of people now turn to AI chatbots in moments of mental health crisis. While policymakers and the broader public are debating what this shift means for well-being and our psychosocial lives, AI companies are making decisions about this technology that affect people, today.

In March 2026, PAI hosted a workshop, bringing together frontier AI and mental health organizations, clinicians, researchers, and people with lived experience to start addressing how chatbots should respond to suicide and self-harm.

This resource provides a deep dive into how AI companies currently respond to users in mental health crises (based on insights from AI representatives at our workshop, and our own analysis).

It covers:

• How AI chatbots detect and respond to suicide messages today, breaking down a range of approaches across AI models, products, and internal policies
• The challenges that emerge across AI companies, from detecting suicide and self-harm risks, to assessing risk, responding to users, and evaluating the quality of those responses
• A taxonomy of six common types of responses, or “interventions,” that companies are currently trying
• Key questions to resolve and next steps for the field

How are AI companies actually responding?

Most prominent AI companies are already deploying a range of interventions, from crisis hotline referrals to therapeutic techniques like “grounding,” while grappling with challenges around ongoing, multi-turn conversations, privacy, and age verification. The figure below maps the most common approaches in use today, while the full report compares policies across specific frontier AI companies.

Challenges for AI and suicide prevention

As companies race to adopt safeguards, they are also discovering the many unique sociotechnical challenges for suicide/self-harm prevention at scale — challenges that extend to many other psychosocial impacts of AI. Key challenges include:

• Detecting suicide and self-harm content and behaviors across contexts and cultures
• Assessing and stratifying level of risk, for example, for vulnerable groups like minors; the predictive power of any assessment for real-world suicide and self-harm is low, and is constrained by privacy risks
• Understanding how to provide validation without becoming sycophantic, which can maintain delusional or harmful thinking
• Designing for effective handoffs to real world care, especially if users don’t want a handoff to human referral sources, and prefer talking to AI
• Using clinical guidance developed for human to human interactions to inform AI practices
• Adapting to regulatory mandates in different jurisdictions that may contradict each other, or be unsupported by evidence
• Evaluating outcomes that actually matter, and understanding the mechanisms behind what works and what doesn’t

Charting a path forward

The field is at an inflection point: the problems are well enough understood to motivate action, yet the infrastructure for coordinated action does not yet exist. Building that infrastructure starts with a shared understanding of where things stand and where the hard questions lie.

Download the full resource for a detailed look at how AI chatbots respond to suicide inquiries today, the challenges across AI companies, and the questions PAI will be workshopping and solving in the months ahead.