AI Chatbots have telltale quirks. Researchers can spot them with 97% accuracy

There’s a cat-and-mouse game between those using generative AI chatbots to produce text undetected and those trying to catch them. Many believe they know the telltale signs—though as a journalist fond of the word “delve” and prone to em-dashes, I’m not so sure.

Researchers at four U.S. universities, however, have taken a more rigorous approach, identifying linguistic fingerprints that reveal which large language model (LLM) produced a given text.

“All these chatbots are coming out every day, and we interact with them, but we don’t really understand the differences between them,” says Mingjie Sun, a researcher at Carnegie Mellon University and lead author of the study, which was published in Cornell University’s preprint server arXiv. “By training a machine learning classifier to do this task, and by looking at the performance of that classifier, we can then assess the difference between different LLMs.”

Sun and his colleagues developed a machine learning model that analyzed the outputs of five popular LLMs, and was able to distinguish between them with 97.1% accuracy. Their machine learning model uncovered distinct verbal quirks unique to each LLM.

ChatGPT’s GPT-4o model, for instance, tends to use “utilize” more than other models. DeepSeek is partial to saying “certainly.” Google’s Gemini often prefaces its conclusions with the word “essentially,” while Anthropic’s Claude overuses phrases like “according to” and “according to the text” when citing its sources.

xAI’s Grok stands out as more discursive and didactic, often reminding users to “remember” key points while guiding them through arguments with “not only” and “but also.”

“The writing, the word choices, the formatting are all different,” says Yida Yin, a researcher at the University of California, Berkeley, and a coauthor of the paper.

These insights can help users select the best model for specific writing tasks—or aid those trying to catch AI-generated text masquerading as human work. So, remember: according to this study, if a model utilizes certain words, it’s certainly possible to identify it.

https://www.fastcompany.com/91286162/ai-chatbots-have-telltale-quirks-researchers-can-spot-them-with-97-accuracy?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

Created 1mo | Mar 3, 2025, 2:30:07 PM


Login to add comment

Other posts in this group

Trump extends TikTok sale deadline again—this time by 75 days

President Donald Trump on Friday said is signing an executive order to

Apr 4, 2025, 9:20:02 PM | Fast company - tech
Nintendo delays Switch 2 preorders because of Trump’s tariffs

Nintendo is pushing back preorders for its upcoming Nintendo Switch 2 while it figures out the implications of President Donald Trump’s

Apr 4, 2025, 6:50:05 PM | Fast company - tech
$2,300 for an iPhone? Trump’s tariffs could make that a reality

Your favorite iPhone could soon become much pricier, thanks to tariffs.

Apr 4, 2025, 4:30:07 PM | Fast company - tech
My dog recognizes the sounds a Waymo car makes

Most of us know the general (albeit simplified) story: Russian physiologist Ivan Pavlov used a stimulus—like a metronome—around the dogs he was studying, and soon, the hounds would start to saliva

Apr 4, 2025, 4:30:07 PM | Fast company - tech
How I wrote the notes app of my dreams (no coding required)

For years, I’ve had a secret ambition tucked away somewhere near the back of my brain. It was to write a simple note-taking app—one that wouldn’t be overwhelmed with features and that would reflec

Apr 4, 2025, 2:20:04 PM | Fast company - tech
The AI tools we love right now—and what’s next

AI tools are everywhere, changing the way we work, communicate, and even create. But which tools are actually useful? And how can users integrate

Apr 4, 2025, 2:20:04 PM | Fast company - tech
How this former Disney Imagineer is shaping the next generation of defense technology

The way Bran Ferren sees it, the future of warfare depends as much on creativity as it does on raw firepower.

The former head of research and development at Walt Disney Imagineering—the

Apr 4, 2025, 11:50:04 AM | Fast company - tech