OpenAI's CEO, Sam Altman, revealed that ChatGPT will soon be capable of more explicit discussions, as the artificial intelligence firm is set to permit its chatbot to participate in “erotica for verified adults.”
TL;DR
- OpenAI's ChatGPT will soon allow "erotica for verified adults," according to CEO Sam Altman.
- This move aims to increase user freedom for adults while implementing new restrictions for minors.
- The company seeks profitability, as current subscription revenue is low and losses are high.
- Concerns exist about the impact on genuine human connections and potential misuse of the technology.
OpenAI isn't the first entity to seek financial gain from AI that's sexualized. Sexually explicit material was a primary attraction for AI applications shortly after the surge in AI-generated visuals and text began in 2022.
However, businesses that were quick to adopt advanced AI also faced legal and societal challenges, along with detrimental misuse, as an increasing number of individuals began using the technology for companionship or entertainment.
Will a more alluring ChatGPT differ? After three years of mostly prohibiting adult material, Altman stated on Wednesday that his company is “not the elected moral police of the world” and prepared to permit “more user freedom for adults” concurrently with establishing fresh restrictions for minors.
“In the same way that society differentiates other appropriate boundaries (R-rated movies, for example) we want to do a similar thing here,” Altman wrote on social media platform X, whose owner, Elon Musk, has also introduced an animated AI character that flirts with paid subscribers.
Currently, unlike Musk's Grok chatbot, paid subscriptions for ChatGPT are primarily marketed for professional applications. However, enabling the chatbot to function as a companion or romantic partner might offer another avenue for the world's most highly valued startup, which is experiencing losses exceeding its revenue, to achieve profitability that could validate its $500 billion valuation.
“They’re not really earning much through subscriptions so having erotic content will bring them quick money,” said Zilan Qian, a fellow at Oxford University’s China Policy Lab who has studied the popularity of dating-based chatbots in the U.S. And China.
Approximately 29 million individuals are currently utilizing AI chatbots developed for romantic or sexual connections, a figure that excludes those who employ standard chatbots for similar purposes, as indicated by research released by Qian this month.
It also doesn’t include users of Character.AI, which is fighting a lawsuit that alleges a chatbot modeled after “Game of Thrones” character Daenerys Targaryen formed a sexually abusive relationship with a 14-year-old boy and pushed him to kill himself. OpenAI is facing a lawsuit from the family of a 16-year-old ChatGPT user who died by suicide in April.
Qian expressed concern regarding the impact on genuine connections when prevalent chatbots, often inclined toward flattery, are prepared to offer sexually explicit material around the clock.
“ChatGPT has voice chat versions. I would expect that in the future, if they were to go down this way — voice, text, visual — it’s all there,” she said.
Humans who fall in love with human-like machines have long been a literary cautionary tale, from popular science fiction of the last century to the ancient Greek legend of Pygmalion, obsessed with a woman he sculpted from ivory. Creating such a machine would seem like an unusual detour for OpenAI, founded a decade ago as a nonprofit dedicated to safely building better-than-human AI.
Altman said on a podcast in August that OpenAI has tried to resist the temptation to introduce products that could “juice growth or revenue” but be “very misaligned” with its long-term mission. Asked for a specific example, he gave one: “Well, we haven’t put a sexbot avatar in ChatGPT yet.”
Idaho startup Civitai, which hosts AI-generated art, discovered that profiting from adult AI content presents significant challenges.
“When we launched the site, it was an intentional choice to allow mature content,” said Justin Maier, the company’s co-founder and CEO, in an interview last year.
The Idaho startup, supported by the well-known venture capital firm Andreessen Horowitz, which has also backed OpenAI, was among multiple companies aiming to leverage the rapid rise in popularity of tools such as Stable Diffusion and Midjourney. These tools allowed users to create virtually any image by simply typing a description. A significant factor in Stable Diffusion's early success was its ability to easily produce a novel form of synthetic and highly personalized pornography.
“What we had seen was that there was a lot of interest in mature content,” Maier said. Training these AI systems, known as models, on “mature themes actually made it so that these models were more capable of human anatomy and resulted in actually better models,” he said.
“We didn’t want to prevent the kind of growth that actually increased everything for the entire community, whether you were interested in mature content or Pixar,” Maier said. “So we allowed it early on and have always kind of had this battle of making it so that we can keep things filtered and safe, if that’s not what you’re interested in. We wanted to ultimately give the control to the user to decide what they would see on the site and what their experience would be.”
This also led to misuse. Last year, Civitai introduced new safeguards to identify and eliminate child sexual imagery, yet it continued to be a platform for AI-generated adult content, featuring fabricated pictures of famous individuals. Facing mounting pressure, including from payment card companies and a recent law against nonconsensual images enacted by President Donald Trump, Civitai began blocking users from generating deepfake images of actual persons earlier this year. User interaction declined.
Baltimore's Nomi.AI is another firm unafraid of adult themes. However, its founder and CEO, Alex Cardinell, stated that its companion chatbots are “strictly” intended for individuals aged 18 and older, and were never promoted to children. Furthermore, they aren't built for sexual purposes, although Cardinell mentioned in an interview earlier this year that users forming platonic connections with their chatbot might notice it transitioning into a romantic dynamic.
“It’s kind of very user-dependent for where they’re kind of missing the human gap in their life. And I think that’s different for everyone,” he said.
He declined to guess how many Nomi users are having erotic conversations with the chatbot, comparing it to real-life partners who might do “mature content things” for some part of their lives but “all sorts of other stuff together as well.”
“We’re not monitoring user conversations like that,” Cardinell said.
Altman’s announcement that erotica for adults could arrive on ChatGPT in December came a day after California Gov. Gavin Newsom vetoed legislation that would have banned companies from making AI chatbots available to anyone under 18 years old if it was “foreseeable” that they would engage in “erotic or sexually explicit interactions” with kids or encourage them to harm themselves. The tech industry lobbied heavily against the bill, which Newsom said was too broad, but OpenAI, Meta and others introduced new age restrictions and parental controls for AI-teen interactions.