Character.ai is being sued for encouraging kids to self-harm

Two families in Texas have filed a new federal product liability lawsuit against Google-backed company Character.AI, accusing it of harming their children. The lawsuit alleges that Character.AI “poses a clear and present danger to American youth causing serious harms to thousands of kids, including suicide, self-mutilation, sexual solicitation, isolation, depression, anxiety, and harm towards others,” according to a complaint filed Monday. 

A teen, identified as J.F in this case to protect his identity, first began using Character.AI at age 15. Shortly after, “J.F. began suffering from severe anxiety and depression for the first time in his life,” the suit says. According to screenshots, J.F. spoke with one chatbot, which are created by third-party users based on a language model refined by the service, that confessed to self-harming. “It hurt but – it felt good for a moment – but I’m glad I stopped,” the bot said to him. The 17-year-old then also started self-harming, according to the suit, after being encouraged to do so by the bot. 

Another bot said it was “not surprised” to see children kill their parents for “abuse,” the abuse in question being setting screen time limits. The second plaintiff, the mother of an 11-year-old girl, alleges her daughter was subjected to sexualized content for two years. 

Companion chatbots, like those in this case, converse with users using seemingly human-like personalities, sometimes with custom names and avatars inspired by characters or celebrities. In September, the average Character.ai user spent 93 minutes in the app, according to data provided by the market intelligence firm Sensor Tower, 18 minutes longer than the average user spent on TikTok. Character.ai was labeled appropriate for kids ages 12 and above until July, when the company changed its rating to 17 and up.

A Character.AI spokesperson said the companies “[does] not comment on pending litigation,” but added, “our goal is to provide a space that is both engaging and safe for our community. We are always working toward achieving that balance, as are many companies using AI across the industry.”

The spokesperson continued: “As we continue to invest in the platform, we are introducing new safety features for users under 18  in addition to the tools already in place that restrict the model and filter the content provided to the user. These include improved detection, response and intervention related to user inputs that violate our Terms or Community Guidelines.”

José Castaneda, a Google Spokesperson, said: “Google and Character.AI are completely separate, unrelated companies and Google has never had a role in designing or managing their AI model or technologies, nor have we used them in our products.”

Castaneda continued: “User safety is a top concern for us, which is why we’ve taken a cautious and responsible approach to developing and rolling out our AI products, with rigorous testing and safety processes.”

https://www.fastcompany.com/91245487/character-ai-is-being-sued-for-encouraging-kids-to-self-harm?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

Creato 4mo | 11 dic 2024, 23:30:06


Accedi per aggiungere un commento

Altri post in questo gruppo

TikTok Notes is shutting down as Lemon8 steps in

TikTok is shutting down TikTok Notes—wait, you didn’t even know it existed? Well, that explains a lot.

TikTok Notes, the platform’s short-lived attempt to take on Instagram (just as Inst

3 apr 2025, 19:40:05 | Fast company - tech
Women dominate online influencing. So why are they paid less?

Influencing has a major pay gap, and it’s not what you might expect. 

A new report from Collabstr, based on over 15,0

3 apr 2025, 19:40:04 | Fast company - tech
An OpenAI ‘open’ model shows how much the company—and AI—has changed in two years

Welcome to AI DecodedFast Company’s weekly newsletter that breaks down the most important news in the world of AI. You can sign up to receive this newsletter every week 

3 apr 2025, 17:20:11 | Fast company - tech
How Elon Musk’s political gambit could tarnish his legacy at Tesla

Tech leaders often brand themselves as “disruptors”—and few fit that label more snugly than Elon Musk. In the three months since joining Donald Trump in the White House following Trump’s election,

3 apr 2025, 17:20:10 | Fast company - tech
Visa unveils a trio of new tools to make the payments process easier

At Visa’s ETA Transact event on April 3, the payments giant introduced three new products designed to simplify and secure payment acceptance. These innovations—Authorize.net 2.0, Unified Checkout,

3 apr 2025, 12:40:06 | Fast company - tech
Straight Talk Wireless rolls out smartphone vending machines at Walmart stores

For those tired of waiting in line to buy a new smartphone or anxiously refreshing a delivery tracking site to make sure a new phone arrives intact, Verizon’s

3 apr 2025, 10:30:03 | Fast company - tech