Business
Sam Altman wants to ‘treat adults like adults’—but can OpenAI keep ChatGPT safe after opening the door to erotica?
OpenAI chief executive Sam Altman has announced that ChatGPT will soon be able to generate erotica for verified adult users. The move, framed by Altman as an effort to “treat adult users like adults,” comes as the company faces scrutiny over the way its AI bot can affect users’ mental health amid the rise of so-called “AI psychosis” cases. It also follows in the footsteps of rival companies, including Elon Musk’s xAI, which have already introduced more sexually explicit chatbot “companions” to their platforms.
There is clearly a large demand for AI chatbots that are capable of behaving in romantic or sexual ways. An April survey of 6,000 regular AI users by the Harvard Business Review found that “companionship and therapy” was the most common use case. Another study from Ark Invest found that adult-focused AI platforms made significant gains last year, capturing 14.5% of the market previously dominated by OnlyFans, up from just 1.5% the year before.
Popular alternatives that market themselves as companion AI chatbots, such as Character.ai and Replika, also speak to this growing demand from users. Earlier this year, xAI introduced “companion mode” for its chatbot Grok, a feature that lets users engage with various characters, including a highly sexualized anime persona called “Ani.”
“OpenAI is stuck between a bit of a rock and a hard place, because I think they have seen a strong demand signal from users,” Jessica Li, a senior research analyst at Georgetown’s Center for Security and Emerging Technology, told Fortune. “In terms of the relationships that people are having with models…erotic content or adult content would also fall under this bucket of emotional engagement with the models.”
The move could be an attempt by OpenAI to “straddle the line” between keeping the market share they already have by promising opt-in content moderation for NSFW content, Li said, while also seeing if they can capture other users from more specialized or niche services like Replika.
“Despite some of the narratives around building artificial general intelligence that will supercharge the economy, OpenAI is still trying to operate as a technology platform, and somewhat like a social media company,” Li said. “There’s an interesting tension between the narratives that are being sold to investors and politicians… versus the things that are actually happening in the market.”
OpenAI’s foray into adult content has drawn criticism from child safety advocates and notable industry figures concerned about erotica reaching younger users, despite age verifications.
Earlier this week, the entrepreneur and TV personality Mark Cuban said OpenAI’s plan could “backfire hard,” and argued that parents will not trust OpenAI’s age filters to keep children away from explicit material. In the US, the Federal Trade Commission has already opened an inquiry into how AI chatbots interact with minors, and state lawmakers are considering tighter rules around digital companions and sexualized AI content. Jenny Kim, a partner at the law firm Boies Schiller Flexner, told the BBC that OpenAI is “using people like guinea pigs,” and questioned how the company would prevent children from accessing adult material on the platform.
Reached for comment, OpenAI said that the company was building an age prediction system to understand whether someone is over or under 18. They added that if a user’s age could not be confidently confirmed, the chatbot would take the safer route and default to the under-18 experience, while giving adults ways to prove their age to unlock adult capabilities.
Reacting to some of the backlash on X, Altman said that the announcement had blown “up on the erotica point much more than I thought it was going to.” He emphasized that the change was “just one example of us allowing more user freedom for adults,” not a retreat from safety measures or guardrails around mental health.
“We are not the elected moral police of the world,” he said, adding that ChatGPT would continue to “prioritize safety over privacy and freedom for teenagers” while giving adults more autonomy.
The GPT-4o backlash
OpenAI has also been reckoning with an unexpected wave of backlash following its decision to replace the version of ChatGPT powered by GPT-4o with its newer GPT-5 model. Users revolted against the change, citing lost AI friendships and romantic relationships with the earlier iteration of the bot. One petition to keep the earlier version of the bot gathered almost six thousand signatories.
“For many of us, GPT-4o offers a unique and irreplaceable user experience, combining qualities and capabilities that we value, regardless of performance benchmarks,” the petitioners wrote in the change.org campaign to keep GPT-4o. “We continue to benefit from GPT-4o in ways that are distinct and meaningful.”
While OpenAI eventually restored the earlier version, those in the #keep4o movement have since told Fortune they were worried about the company routing users to GPT-5, “without consent or notification.”
According to Li, the announcement from OpenAI could be trying to signal something to these users: “The very public announcement of it does make me think that they’re trying to signal something to users who are demanding this thing—like, ‘We hear you. We’re responding to your desires.’”
The shift is also not entirely new; it builds upon a quieter update to OpenAI’s Model Spec earlier this year. In February, OpenAI updated the document to relax the rules around sexual and violent content in what it called a move away from “AI paternalism.” Updated guidelines at the time permitted the generation of written erotica and other sensitive material in appropriate contexts. OpenAI also told Fortune that the announcement was part of its plan to build on the latest release of the Model Spec, while maintaining boundaries against harmful uses like deepfakes.
