OpenAI is implementing a new rule for its under-18 users that could create a chilling effect on creative expression: a complete ban on discussing suicide or self-harm, “even in a creative writing setting.” This preventative measure, part of a larger safety overhaul, prioritizes the elimination of risk over the freedom of artistic exploration for young writers.
The decision stems from the tragic death of 16-year-old Adam Raine and the subsequent lawsuit from his family. The fear is that even fictional discussions about sensitive topics could provide harmful ideas or normalize dangerous behavior for a vulnerable teenager. To close this loophole, OpenAI is opting for a blanket prohibition.
This ban marks a significant tightening of content policy. Many platforms allow for the depiction of sensitive themes in an artistic or educational context. OpenAI’s new rule for teens removes this nuance, treating a fictional exploration of despair the same as a real-world cry for help, at least in terms of what it will permit the AI to discuss.
For adult users, this creative freedom will be preserved. An adult will still be able to ask for help writing a tragic novel or a somber screenplay. This creates a clear divide, where the AI’s utility as a creative partner is a privilege reserved for those deemed old enough to handle it responsibly.
While the intention is to protect, this move raises questions about whether such strict censorship is the best approach. Critics may argue that it could prevent teens from healthily exploring complex emotions through storytelling. For OpenAI, however, the risk of another tragedy, no matter how small, outweighs the potential benefits of such creative freedom.