Nvidia’s new AI model Fugatto can generate music and audio

Nvidia on Monday showed a new artificial intelligence model for generating music and audio that can modify voices and generate novel sounds — technology aimed at the producers of music, films and video games.

Nvidia, the world’s biggest supplier of chips and software used to create AI systems, said it does not have immediate plans to publicly release the technology, which it calls Fugatto, short for Foundational Generative Audio Transformer Opus 1.

It joins other technologies shown by startups such as Runway and larger players such as Meta Platforms that can generate audio or video from a text prompt.

Santa Clara, California-based Nvidia’s version generates sound effects and music from a text description, including novel sounds such as making a trumpet bark like a dog.

What makes it different from other AI technologies is its ability to take in and modify existing audio, for example by taking a line played on a piano and transforming it into a line sung by a human voice, or by taking a spoken word recording and changing the accent used and the mood expressed.

“If we think about synthetic audio over the past 50 years, music sounds different now because of computers, because of synthesizers,” said Bryan Catanzaro, vice president of applied deep learning research at Nvidia. “I think that generative AI is going to bring new capabilities to music, to video games and to ordinary folks that want to create things.”

While companies such as OpenAI are negotiating with Hollywood studios over whether and how the AI could be used in the entertainment industry, the relationship between tech and Hollywood has become tense, particularly after Hollywood star Scarlett Johansson accused OpenAI of imitating her voice.

Nvidia’s new model was trained on open-source data, and the company said it is still debating whether and how to release it publicly.

“Any generative technology always carries some risks, because people might use that to generate things that we would prefer they don’t,” Catanzaro said. “We need to be careful about that, which is why we don’t have immediate plans to release this.”

Creators of generative AI models have yet to determine how to prevent abuse of the technology such as a user generating misinformation or infringing on copyrights by generating copyrighted characters.

OpenAI and Meta similarly have not said when they plan to release to the public their models that generate audio or video.

—Stephen Nellis, Reuters

https://www.fastcompany.com/91235722/nvidia-ai-model-generates-music-audio-sounds?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

Created 5mo | Nov 25, 2024, 6:10:04 PM


Login to add comment

Other posts in this group

Is social media hurting teens’ mental health? It’s complicated

Social media is terrible for teens’ mental health—or is it?

At the same time that

Apr 28, 2025, 6:10:07 AM | Fast company - tech
3 quick, easy AI chatbot prompts that can help you do your job better

Fun fact: The saying “work smarter, not harder” is coming up on its 100th birthday. Coined

Apr 28, 2025, 6:10:06 AM | Fast company - tech
Is Apple falling behind on hardware?

If you’ve followed Apple for any length of time, you’ve no doubt come across the notion that the company doesn’t rush into adopting cutting-

Apr 27, 2025, 11:30:07 AM | Fast company - tech
This free audio enhancer will totally transform your voice memos

Every now and then, you run into a tool that truly wows you.

It’s rare—especially nowadays, when everyone and their cousin is coming out with overhyped AI-centric codswallop tha

Apr 26, 2025, 12:20:10 PM | Fast company - tech
Elon Musk’s Trump gamble is costing him bigly

Tesla released its quarterly earnings report on Tuesday, its first since the company’s chief executive, Elon Musk, took up residence in the Trump White House and immediately began trying to fire f

Apr 26, 2025, 12:20:09 PM | Fast company - tech
Say goodbye to cheap versions of Ozempic and Wegovy

There’s never a dull day in the world of weight-loss medication. This week brought new restrictions on compounded GLP-1 medication, the cheaper, copycat versions of brand-name drugs that tel

Apr 26, 2025, 12:20:08 PM | Fast company - tech
Why Apple needs Tim Cook more than ever in the age of Trump

In December 2023, I wrote an article exploring Apple CEO Tim Cook’s most likely successors, because t

Apr 26, 2025, 10:10:03 AM | Fast company - tech