Google’s Jigsaw launches an open source anti-harassment tool for women

Jigsaw, a unit within Google devoted to developing technology to address global threats like censorship and disinformation, just launched an open source tool for combatting harassment on Twitter. It’s intended to help female journalists, activists, and public figures, many of whom have been the victim of harassment on the social media platform. The new program, called Harassment Manager, is available on Github. The tool is an application programming interface (API), meaning it allows apps and other programs to work in tandem; the API uses machine learning to highlight abusive posts and coordinated harassment campaigns, enabling Twitter users to better document such instances. The API also lets users automatically block and mute accounts that appear in the harassment report it generates. To create its Harassment Manager, Jigsaw interviewed 27 female journalists and activists to understand what tools would be most helpful to them in responding to online harassment. “We brought in journalists to come and sit with us and design together in workshops, and one of the things that we heard consistently is, ‘When I experience harassment, the first thing I do is I document that harassment,'” says Jigsaw engineering manager Lucy Vasserman. Victims need to document abuse in order to flag it to platforms or, in more serious incidents, to law enforcement. Already, Thompson Reuters Foundation has committed to productizing the Harassment Manager and rolling it out to the thousands of journalists it works with globally as part of its work to defend free and open media. The interface will be in English, though the machine learning can read tweets in 17 languages, says Vasserman, including Arabic, Chinese, Czech, Hindi, Indonesian, Japanese, Korean, Polish, Portuguese, and Russian. The announcement comes at a time when women are asking for more protections against harassment online. Taylor Lorenz, a reporter at the Washington Post (formerly with the New York Times), who covers the creator economy, has spoken out repeatedly about the damaging effects of online harassment. “This person has actually commandeered thousands of accounts and is using them to blast out the same crazy lies in an attempt to smear me. He is using bots or hacked profiles to do this,” she tweeted about a recent attack. “These types of modern day smear campaigns are what newsrooms need to prepare for.” Lorenz also called on media organizations to better support journalists in their organization who face these sorts of attacks.

This person has actually commandeered thousands of accounts and is using them to blast out the same crazy lies in an attempt to smear me. He is using bots or hacked profiles to do this. Again, these types of modern day smear campaigns are what newsrooms need to prepare for pic.twitter.com/epsbYSMbwf — Taylor Lorenz (@TaylorLorenz) March 1, 2022

Twitter has long been a toxic place for women. A 2018 Amnesty International report found that, on average, women receive an abusive tweet every 30 seconds. Three years later, 73% of female journalists surveyed in a UNESCO report said they had experienced violence online—a startling figure that the report called “an attack on democratic deliberation and media freedom.” A number of tools have launched in the last several years to help women navigate Twitter. Tracy Chou, who herself suffered years of online abuse, debuted her online harassment tool Block Party last year, which mutes accounts based on a user’s preferences. and quarantines direct messages from those accounts. That same year, Coalition Against Online Violence launched the Online Violence Response Hub, which helps women with everything from doxxing to legal support. Twitter has long been criticized for allowing toxic behavior to run rampant on the platform (former CEO Dick Costolo acknowledged Twitter “sucked” at dealing with trolls in an internal memo back in 2015). Yet it wasn’t until last September that Twitter started beta testing its own automated anti-harassment tool, called Safety Mode, which blocks offensive accounts on behalf of users. Jigsaw’s Harassment Manager tool emerged out of its existing artificial intelligence program, Perspective, which was developed in 2017 to support content moderators. Perspective, which is used by organizations like the New York Times and Reddit, rates the likelihood that a reader perceives a post or comment as toxic. Now with Harassment Manager, Vasserman is hoping to not just help platforms keep toxic content off their sites, but to help individuals who face the brunt of abusive posts online. And if all goes according to plan, the Harassment Manager will go beyond Twitter. “We’ve built it with Twitter for now,” says Vasserman. “But our vision is that others can contribute, and this does not need to work exclusively on Twitter forever. We hope that it can incorporate APIs to connect to more platforms.”

https://www.fastcompany.com/90728747/google-jigsaw-anti-harassment-tool-twitter?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

Creată 3y | 8 mar. 2022, 10:21:08


Autentifică-te pentru a adăuga comentarii

Alte posturi din acest grup

TikTok has a big ADHD problem

ADHD content has been trending on TikTok for a while. Unsurprisingly, much of it has been found to be misleading. 

A study published last week

24 mar. 2025, 16:20:05 | Fast company - tech
Section 230 faces bipartisan repeal effort. Experts say it’s a risky bet

Section 230 of the Communications Decency Act—the provision that protects tech platforms from legal liability for content posted by their users—has long been a point of contention among lawmakers.

24 mar. 2025, 16:20:04 | Fast company - tech
Flexport CEO Ryan Petersen on what business leaders need to know about Trump’s tariffs

Trump’s tariff war is causing vast disruption across international trade. Ryan Petersen, foun

24 mar. 2025, 13:50:06 | Fast company - tech
How ‘The Matrix’ helped build—and break—beleaguered production company Village Roadhouse

The name Village Roadshow might not ring a bell with every moviegoer, but "> the company’s logo almost certainly will. Its shimmering, nested

24 mar. 2025, 11:40:02 | Fast company - tech
Would you pay $4,000 to solve your streaming movie problems?

Let’s say you were spending tens of thousands of dollars to build yourself a fancy home theater. How would you go about actually watching movies in it?

While you could always set u

24 mar. 2025, 09:20:04 | Fast company - tech
These tech companies are building healthier social media habits for kids

The last year has seen a global reckoning with the effects of social media on kids. Australia banned

23 mar. 2025, 12:30:02 | Fast company - tech