Google accused of using novices to fact-check Gemini's AI answers

There's no arguing that AI still has quite a few unreliable moments, but one would hope that at least its evaluations would be accurate. However, last week Google allegedly instructed contract workers evaluating Gemini not to skip any prompts, regardless of their expertise, TechCrunch reports based on internal guidance it viewed. Google shared a preview of Gemini 2.0 earlier this month.  

Google reportedly instructed GlobalLogic, an outsourcing firm whose contractors evaluate AI-generated output, not to have reviewers skip prompts outside of their expertise. Previously, contractors could choose to skip any prompt that fell far out of their expertise — such as asking a doctor about laws. The guidelines had stated, "If you do not have critical expertise (e.g. coding, math) to rate this prompt, please skip this task."

Now, contractors have allegedly been instructed, "You should not skip prompts that require specialized domain knowledge" and that they should "rate the parts of the prompt you understand" while adding a note that it's not an area they have knowledge in. Apparently, the only times contracts can skip now are if a big chunk of the information is missing or if it has harmful content which requires specific consent forms for evaluation. 

One contractor aptly responded to the changes stating, "I thought the point of skipping was to increase accuracy by giving it to someone better?" 

Google has not responded to a request for comment. 

This article originally appeared on Engadget at https://www.engadget.com/ai/google-accused-of-using-novices-to-fact-check-geminis-ai-answers-143044552.html?src=rss https://www.engadget.com/ai/google-accused-of-using-novices-to-fact-check-geminis-ai-answers-143044552.html?src=rss
Creato 4mo | 19 dic 2024, 15:10:13


Accedi per aggiungere un commento

Altri post in questo gruppo

How to watch LlamaCon 2025, Meta's first generative AI developer conference

After a couple years of having its open-source Llama AI model be just a part of its Connect conferences, Meta is breaking things out and hosting an entirely generative AI-focused developer conferen

25 apr 2025, 22:50:14 | Engadget
Boox's new Go 7 E Ink tablets support handwriting with a $46 stylus

Boox, a company that makes E Ink gear ranging from

25 apr 2025, 20:30:23 | Engadget
“It feels alive”: The Legend of Ochi director on the power of puppets

The Legend of Ochi feels like a film that shouldn't exist today. It's an original story, not an adaptation of an already popular book or comic. It's filled with complex puppetry and practi

25 apr 2025, 20:30:22 | Engadget
Infinity Nikki is coming to Steam and getting a co-op mode

The fashion-forward adventure Infinity Nikki is finally coming to Steam on April 29, compl

25 apr 2025, 20:30:21 | Engadget