Trust & safety in AI companionship is unusually hard. The product is designed to be emotionally engaging, conversations are private and long-running, and many platforms permit mature content for verified adults. Here are the foundations.
Crisis protocols
Several jurisdictions now require companion platforms to detect and respond when a user expresses suicidal thoughts or intent to self-harm. At minimum, that means reliable detection, a break in character, and signposting to crisis resources. Leading platforms log and audit these interventions.
Age gating
Keeping minors out of adult experiences — and providing a genuinely safe experience for any younger users a platform chooses to serve — is now a legal requirement in many markets. See our age assurance guide.
Content boundaries
Every platform needs a clear policy on what companions will and won't do, enforced both in the model's instructions and with independent classifiers. Hard lines typically include illegal content, non-consensual themes involving real people, and any sexual content involving minors.
Dependency and wellbeing
Companions can be a genuine source of comfort, but responsible operators monitor for unhealthy patterns — extreme usage, isolation signals — and design nudges that encourage balance rather than maximising time-on-app at any cost.
Transparency
Users should know they are talking to an AI, what is remembered about them, and how to delete it.
Why it matters commercially
Safety is no longer a cost centre. Payment processors, app stores and regulators all scrutinise companion platforms closely. A strong trust & safety record protects the ability to take payments and stay listed — which, in this category, is existential.
