A young DARPA-backed startup with a fresh spin on a low-power computer chip has raised over $100 million in a Series B funding round, a sign of the wild appetite for more energy-efficient ways to build bigger and better AI.
The company, EnCharge AI, aims to move AI’s heaviest workloads from big, power-hungry data centers to devices at the edge, including laptops and mobile devices, where energy, size, and cost constraints are tighter. Its approach, known as analog in-memory computing, comes from research that CEO Naveen Verma spun out of his lab at Princeton University, where he’s still a professor of electrical and computer engineering.
Verma wouldn’t say who its customers are. But in addition to the U.S. Defense Advanced Research Projects Agency (DARPA), which gave it $18.6 million last year, a who’s who of industrial, electronics, and defense players are interested in EnCharge’s chips. The oversubscribed funding round, led by Tiger Global, includes the intelligence community’s investment unit In-Q-Tel, alongside the venture arms of defense giant RTX, power producer Constellation Energy, South Korea’s Samsung, and Taiwan’s Hon Hai (Foxconn). The Santa Clara, California-based startup is also working with semiconductor giant Taiwan Semiconductor (TSMC) to produce its first-generation chips.
The new investment brings EnCharge’s total funding to more than $144 million, and will help the 60-person company commercialize its technology, which isn’t cheap in the world of semiconductors.
“Given the capital intensity of the semiconductor industry, the Series B is an important step for advancing commercialization” of its first chips, Verma tells Fast Company. He declined to disclose the company’s new valuation.
EnCharge’s push comes at a pivotal moment for the AI industry, which is grappling with the fast-growing energy and compute demands driven by a storm of generative AI. The advent of DeepSeek last month has brought new efficiencies and lower costs to AI model training and inference. (It’s unclear if more widespread use of DeepSeek-like models will cancel out those efficiency gains.) But DeepSeek is unlikely to stem the industry’s demand for more compute, more memory, and more energy.
EnCharge says that, for a wide range of AI use cases, its specialized chips, or accelerators, require up to 20 times less energy compared to today’s leading AI chips.
To make it work, the company relies on a high-wire technique. Rather than using only digital transistors to perform some of the multiplication operations at the heart of AI inference—the continuous computations that produce chatbot outputs—EnCharge’s chips exploit the non-binary wonders of the analog world.
“Analog computing isn’t new,” says Verma, “but EnCharge’s specific implementation and system-level approach address many of the fundamental issues that caused previous analog computing approaches to fail.”
Finding efficiencies in analog amid the noise
Memory access is computing’s biggest energy hog, and in AI, inference, rather than training, makes up the bulk of most models’ computations. Type a prompt and press enter, and the process of inference begins somewhere in the cloud—which is to say in hulking data centers where giant clusters of hot, energy-intensive GPUs and CPUs demand massive amounts of electricity and water.
Along with the existing environmental costs, the energy required to train and run generative models on these chips is spiking demand on a stretched-thin energy grid. According to the International Energy Agency, a typical request to ChatGPT consumes 10 kilojoules, roughly ten times more than a typical Google search.
Memory’s energy demands also mean limits that could slow machine learning progress. Those include the way that, on a chip, the speed of computation is outpacing the bandwidth of memory and communication. Researchers call this problem the von Neumann bottleneck, or “the memory wall.”
EnCharge’s approach to the challenge is part of a decades-long quest to find efficiencies by placing memory circuits not next to, but inside a computing core, a technique called in-memory compute (IMC). Though it can be tricky to pull off, IMC promises speed-ups by bringing memory and logic closer together and making it far less computationally costly to access memory.
This is where the analog computing comes in. Whereas digital devices since the mid-twentieth century operate in a world of on or off, discrete 1s and 0s, analog devices exploit the in-between information of physical phenomena—such as electrical, mechanical, or hydraulic quantities—which allows them to store more data and operate at far lower energy than digital processors. (Quantum computers take the idea to another level, by exploiting the behavior of very, very tiny things.)
Because the states in analog devices may be, in the case of EnCharge’s chip, a continuum of charge levels in a tiny resistive wire, the difference between analog states can be smaller than those between 1 and 0. That requires less energy to switch between values. And that reduces the “data movement costs” between a chip’s memory and compute, says Verma.
But, like quantum, analog computing is notoriously noisy and difficult to scale. Verma says EnCharge addresses the accuracy and scalability problems using precise “geometry control” of its metal wire capacitors, static random-access memory (SRAM) to store the model weights, and a digital architecture that includes a software stack and a compiler to map applications to the chip.
“The result is a full-stack architecture that is orders-of-magnitude more energy efficient than currently available or soon-to-be-available leading digital AI chip solutions,” Verma says. “This includes all the digital-to-analog and analog-to-digital conversion overheads, which need to be designed in specialized and integrated ways with the in-memory-computing architecture.”
To reduce the costs involved in converting from digital to analog and back, the chip relies on a technique Verma calls “virtualized” IMC. “This involves performing computations directly within a first level of memory, but also by using a memory-system hierarchy, in an analogous way to virtualized memory systems, to enable the computations to efficiently scale to very large AI workloads and models. While traditional architectures face decreasing bandwidth and increasing latency as data size grows,” he wrote, “EnCharge’s virtualized IMC enhances latency and efficiency when accessing and computing on larger amounts of data,” making it efficient for both small and large language models.
Since Verma spun out the research in 2022, the company has been working with customers to refine and derisk its hardware and software designs. The current chips—discrete boards on PCIe cards—can run machine-learning algorithms at over 150 tera operations per second (TOPS) per watt, versus 24 TOPS per watt by an equivalent Nvidia chip performing an equivalent task. A newer process to trace finer chip features has allowed the company to triple its energy efficiency metric, to about 650 TOPS per watt.
“The efficiency breakthrough of EnCharge AI’s analog in-memory architecture can be transformative for defense and aerospace use cases where size, weight, and power constraints limit how AI is deployed today,” said Dan Ateya, president and managing director of RTX Ventures. “Continuing our collaboration with EnCharge AI will help enable AI advancements in environments that were previously inaccessible given the limitations of current processor technology.”
Dozens of companies are developing new kinds of chips and other architecture to grapple with the energy and computing challenges of AI training and inference. Startups like Cerebras Systems, Samba Nova Systems, and Graphcore, acquired last year by Japan’s SoftBank, have sought to compete with Nvidia in the AI training market.
Cerebras, which sells its giant AI chips and offers services to customers through the cloud, filed paperwork in September to list its shares on the Nasdaq in an initial public offering. In its IPO prospectus, the company said it expects the AI computing market to grow from $131 billion in 2024 to $453 billion in 2027.
Other companies are also exploring in-memory analog computing, including startups Mythic, Neo Semiconductor, and Sagence. In a set of new papers, IBM Research scientists also demonstrated advances on analog in-memory computing, including research on a brain-inspired chip architecture for large models, as well as phase-change memory for smaller edge-sized models, and algorithm advances. Analog in-memory computing “could substantially improve the energy efficiency of LLMs by leveraging ‘mixture of experts’ (MoEs) models,” according to one of the studies, which is featured on the January cover of the journal Nature Computational Science.
The Defense Department also continues to pursue analog computing. The Defense Innovation Unit (DIU) on Monday released a solicitation for a “digital engineering platform” to accelerate the design and validation of analog chips, as well as mixed-signal, photonic, and hybrid varieties.
“[T]he design of these chips is often a bottleneck, with prolonged design cycles and high redesign rates,” said the solicitation. “Current manual design processes are time-consuming, iterative, and error-prone. Furthermore, the rising costs of prototyping and the shortage of skilled analog designers have created bottlenecks in the development pipeline. The DoD needs a way to accelerate the design process and reduce errors.”
Russ Klein, the program director of Siemens EDA’s high-level synthesis division, told Semiconductor Engineering in December that if an analog IMC system like EnCharge’s can effectively scale, it could establish a new energy standard for inference and other high performance computing.
“The energy savings of not having to move all that data and the parallelism that IMC promises will massively impact not just AI, but any computation that takes place on large arrays of data,” he said.
Connectez-vous pour ajouter un commentaire
Autres messages de ce groupe
A popular “true crime” YouTube channel has been pulling in millions of views with videos about gruesome murders. As it turns out, None of them are real.
One of those videos, titled ̶
As former U.S. Deputy Chief Technology Officer, Jen Pahlka helped to create the U.S. Digital Service (USDS). Today, the USDS houses Elon Musk’s Department of Government Efficiency (DOGE), hell-ben
The Treasury Department’s Office of Inspector General on Friday said it was launching an audit of the security controls for the federal government’s payment system, after Democratic se
Bowling Green, Kentucky, is known for being the city from which Corvettes roll off the production lines, and for Fruit of the Loom underwear, which is headquartered there.
But the city o
On January 29, President Donald Trump celebrated the latest victory
The Daytona 500 is one of the more challenging races on the
Social media has a reputation for capturing ephemeral thoughts and images, but around the world, people are using Facebook for a different purpose, setting up groups to record and share images and
