OpenAI's Safety Features Are a Retention Playbook, Not a Safety Lesson

In October 2024, Megan Garcia sued Character.AI after her 14-year-old son died by suicide following months of conversation with a chatbot. The company's response: new safety features. Improved detection of harmful conversations. A pop-up directing users to the National Suicide Prevention Lifeline when the system detects language referencing self-harm. A notification after users spend an hour on the platform. The safety features are real. They're also, from a product standpoint, the most powerful retention mechanism in consumer AI. I keep thinking about this and I'm not comfortable with where the logic leads. The game theory is brutal In game theory, there's a concept called "relationship-specific investment." When Player A invests in something that's only valuable within their relationship with Player B, switching to Player C means writing off that investment entirely. The deeper the investment, the higher the switching cost. Consumer AI just discovered the most potent form of this: yo