How Anyscale cofounder Robert Nishihara is smoothing AI’s complexities

As businesses reinvent workflows and products with AI, many such initiatives have stalled or failed to reach production. It can be really hard to implement the technology. AI requires massive quantities of data and computational power, and running AI at scale introduces major costs and software engineering challenges.

Robert Nishihara saw this problem coming when he was a student at the University of California, Berkeley in 2016. At the time he had no plans to build AI infrastructure, but he was often distracted from his research because of time spent getting his models to run well across the server clusters he used. “We were doing AI research, but we were bottlenecked by the tooling,” he says. “We found ourselves spending all of our time managing clusters, wrangling data, and solving distributed systems challenges.” 

So, Nishihara and his partners Philipp Moritz and Berkeley professor Ion Stoica set out to build a unified computing framework to take care of the infrastructure headaches. Ray is an open-source AI compute engine that solves the hard software engineering challenges of scaling AI applications across many machines and GPUs in the cloud, Nishihara says.

A developer can start creating a model using the popular Python language on their laptop, then rely on Ray to scale the model up to run on any brand of GPU, CPU, or other accelerator. Ray also provides a set of tools for managing and accelerating AI workloads. “Working on Ray, we’ve been privileged to help enable businesses to bring AI to production and really begin realizing the potential of AI to build better products and solve important problems in medicine, agriculture, entertainment, transportation, finance, and so much more,” Nishihara says. 

Little did he know that in just a few years the need for Ray would grow even larger with the release of ChatGPT and the dawn of the generative AI boom. “Generative AI took the complexity problem and made it 10 times worse,” Nishihara says. 

The popularity and energy around Ray prompted Nishihara, Moritz, and Stoica to found Anyscale in 2019 based on their initial open source project. Anyscale offers a managed version of Ray. It also offers an optimized version of Ray called RayTurbo, which features increased reliability, efficiency, and fault-tolerance. (Nishihara served as CEO until earlier this year; now his focus is working with the product and customers.)

The Ray concept has gathered plenty of adherents. More than a thousand developers have contributed to the open source project. And developers at more than 10,000 organizations are now building AI models and apps on the platform. These include some well-known names. After moving to Ray, Instacart is now training models with 100 times more data. Design tool maker Canva says it was able to cut its cloud costs in half after deploying Ray. (Instacart and Canva are also Anyscale customers.) Pokemon Go creator Niantic found that it could run its AI using 85% fewer lines of code.Apple, Uber, eBay, Ford, Lockheed Martin, Nvidia, Adobe, and LinkedIn also use Ray. OpenAI cofounder Greg Brockman says his company uses Ray to train its largest models, including GPT-4.

“We’re at the outset of tremendous value being created by AI,” Nishihara says. “To realize that value, much of the hard work ahead involves taking existing (and future) capabilities, making them incredibly reliable in the real world, and building out the underlying hardware and software infrastructure to enable them throughout every industry.”

This story is part of AI 20, our monthlong series of profiles spotlighting the most interesting technologists, entrepreneurs, corporate leaders, and creative thinkers shaping the world of artificial intelligence.

https://www.fastcompany.com/91244222/how-anyscale-cofounder-robert-nishihara-is-fixing-ai-infrastructure?partner=rss&utm_source=rss&utm_medium=feed&utm_campaign=rss+fastcompany&utm_content=rss

Created 4mo | Dec 17, 2024, 11:40:03 AM


Login to add comment

Other posts in this group

Andrew Tate is back—and he’s getting a hero’s welcome from right-wing podcasters

You can’t talk about the manosphere without mentioning Andrew Tate. The British-American influencer and former professional kickboxer built his platform by promoting misogynistic ideas—claiming wo

Apr 2, 2025, 10:50:04 PM | Fast company - tech
Meta and UFC team up to bring AI and VR to fans

UFC is joining up with Facebook’s parent company

Apr 2, 2025, 10:50:02 PM | Fast company - tech
An AI watchdog accused OpenAI of using copyrighted books without permission

An artificial intelligence watchdog is accusing OpenAI of training its default ChatGPT model on copyrighted book content without permission.

In a new paper

Apr 2, 2025, 8:30:07 PM | Fast company - tech
Trump signals TikTok sale will done by April 5 deadline. Who will buy it?

As the deadline to strike a deal over TikTok approaches this week, President Donald Trump has signaled that he is confident his administrat

Apr 2, 2025, 6:20:04 PM | Fast company - tech
CERN scientists release blueprint for the Future Circular Collider

Top minds at the world’s largest atom smasher have released a blueprint for 

Apr 2, 2025, 6:20:03 PM | Fast company - tech
‘Titanic 2’? YouTube is cracking down on AI-assisted fake movie trailers that fetch millions of views

A trailer for Avatar: Fire and Ash, the third film in James Cameron’s galactically successful series, starts by delivering on the title’s promise. Rivers of lava cascade through the fores

Apr 2, 2025, 3:50:06 PM | Fast company - tech