A group of internet businesses, including Roblox, Google, OpenAI, and Discord, have cofounded a nonprofit called Robust Open Online Safety Tools (ROOST).
The new organization will fund free, open-source tools for online businesses to promote online safety, says Naren Koneru, Roblox’s vice president of engineering, trust, and safety. The move follows years of efforts by Roblox to restrict inappropriate messaging on its platform, which is widely used by children and has at times come under fire for not doing enough to combat sexual content and adult sexual predators.
And while human moderators are part of that equation, AI and automation have become critical for intercepting real-time unwanted messages across the platform’s 85 million daily active users, Koneru says.
“These decisions need to happen within milliseconds,” he says.
Among the tools Roblox has developed is an open-source AI model that analyzes audio clips to detect profanity, racism, bullying, sexting, and other disallowed content. The model was released to the public last July, available on GitHub and the AI platform Hugging Face, and it’s since been downloaded more than 20,000 times. The company has since developed a new version of the model, with support for additional languages including Spanish, French, German and Japanese, as well as additional infrastructure for fine-tuning the model to particular needs. That will likely be open sourced by the end of the first quarter of 2025, and the company anticipates unveiling other open-source tools for classifying content later in the year.
By contributing to ROOST (and acting as co-chair of a technical advisory committee), Roblox will also be able support such open-source efforts, aiming to create AI models that can be used by organizations of all sizes to moderate content, especially around child safety.
“While large companies like us can invest in systems like this, ” says Koneru, “if you’re a small game developer and you want to build all these safety systems, it’s almost next to impossible today to do it right.”
Some AI systems may even be hosted by ROOST itself, allowing outside companies to easily integrate them via API calls rather than handling complex infrastructure, Koneru says.
“They may actually not just open-source models, but they will also possibly run these hosted services where you can just call an API, as opposed to even worrying about all of these nitty-gritty details of, how do you run this model in a super efficient way,” he says.
ROOST may also release open-source infrastructure for labeling sample training data, like examples of allowed and disallowed content, and manage how it’s used to train and refine AI systems. That includes technology to effectively manage large-scale human moderation efforts and ensure consistency in decisions around rules (to ensure AI models are trained on reliable samples).
In addition to the for-profit companies, ROOST is backed by a variety of philanthropic organizations, including the Future of Online Trust and Safety Fund, Knight Foundation, AI Collaborative and the McGovern Foundation. It’s raised over $27 million to support its first four years of operation.
Jelentkezéshez jelentkezzen be
EGYÉB POSTS Ebben a csoportban
For well over a decade now, consumers have been used to new iPhones coming out in the fall, like clockwork. However, according to a series of reports, Apple may be planning to change its iPhone re
Booking travel has become a bit of a game—especially if you want to get the best possible prices and avoid getting ripped off.
That’s because hotels and airlines have developed the lovel
Uber is facing internal staff unrest as it attempts to implement a three-day-per-week return to office (RTO) mandate and stricter sabbatical eligibility.
An all-hands meeting late
A study has confirmed what we all suspected: “K” is officially the worst text you can send.
It might look harmless enough, but this single letter has the power to shut down a conversatio
SoundCloud is facing backlash after creators took to social media to complain upon discovering that the music-sharing platform uses uploaded music to train its AI systems.
According to S
The Trump administration on Thursday proposed a multibillion-dollar overhaul of a
