Thanks to the artificial intelligence boom, new data centers are springing up as quickly as companies can build them. This has translated into huge demand for power to run and cool the servers inside. Now concerns are mounting about whether the U.S. can generate enough electricity for the widespread adoption of AI, and whether our aging grid will be able to handle the load.
“If we don’t start thinking about this power problem differently now, we’re never going to see this dream we have,” said Dipti Vachani, head of automotive at Arm. The chip company’s low-power processors have become increasingly popular with hyperscalers like Google, Microsoft , Oracle and Amazon — precisely because they can reduce power use by up to 15% in data centers.
Nvidia‘s latest AI chip, Grace Blackwell, incorporates Arm-based CPUs it says can run generative AI models on 25 times less power than the previous generation.
“Saving every last bit of power is going to be a fundamentally different design than when you’re trying to maximize the performance,” Vachani said.
This strategy of reducing power use by improving compute efficiency, often referred to as “more work per watt,” is one answer to the AI energy crisis. But it’s not nearly enough.
One ChatGPT query uses nearly 10 times as much energy as a typical Google search, according to a report by Goldman Sachs. Generating an AI image can use as much power as charging your smartphone.
This problem isn’t new. Estimates in 2019 found training one large language model produced as much CO2 as the entire lifetime of five gas-powered cars.
The hyperscalers building data centers to accommodate this massive power draw are also seeing emissions soar. Google’s latest environmental report showed greenhouse gas emissions rose nearly 50% from 2019 to 2023 in part because of data center energy consumption, although it also said its data centers are 1.8 times as energy efficient as a typical data center. Microsoft’s emissions rose nearly 30% from 2020 to 2024, also due in part to data centers.
And in Kansas City, where Meta is building an AI-focused data center, power needs are so high that plans to close a coal-fired power plant are being put on hold.
Hundreds of ethernet cables connect server racks at a Vantage data center in Santa Clara, California, on July 8, 2024.
Katie Tarasov
Chasing power
There are more than 8,000 data centers globally, with the highest concentration in the U.S. And, thanks to AI, there will be far more by the end of the decade. Boston Consulting Group estimates demand for data centers will rise 15%-20% every year through 2030, when they’re expected to comprise 16% of total U.S. power consumption. That’s up from just 2.5% before OpenAI’s ChatGPT was released in 2022, and it’s equivalent to the power used by about two-thirds of the total homes in the U.S.
CNBC visited a data center in Silicon Valley to find out how the industry can handle this rapid growth, and where it will find enough power to make it possible.
“We suspect that the amount of demand that we’ll see from AI-specific applications will be as much or more than we’ve seen historically from cloud computing,” said Jeff Tench, Vantage Data Center’s executive vice president of North America and APAC.
Many big tech companies contract with firms like Vantage to house their servers. Tench said Vantage’s data centers typically have the capacity to use upward of 64 megawatts of power, or as much power as tens of thousands of homes.
“Many of those are being taken up by single customers, where they’ll have the entirety of the space leased to them. And as we think about AI applications, those numbers can grow quite significantly beyond that into hundreds of megawatts,” Tench said .
Santa Clara, California, where CNBC visited Vantage, has long been one of the nation’s hot spots for clusters of data centers near data-hungry clients. Nvidia’s headquarters was visible from the roof. Tench said there’s a “slowdown” in Northern California due to a “lack of availability of power from the utilities here in this area.”
Vantage is building new campuses in Ohio, Texas and Georgia.
“The industry itself is looking for places where there is either proximate access to renewables, either wind or solar, and other infrastructure that can be leveraged, whether it be part of an incentive program to convert what would have been a coal-fired plant into natural gas, or increasingly looking at ways in which to offtake power from nuclear facilities,” Tench said.
Vantage Data Centers is expanding a campus outside Phoenix, Arizona, to offer 176 megawatts of capacity
Vantage Data Centers
Hardening the grid
The aging grid is often ill-equipped to handle the load even where enough power can be generated. The bottleneck occurs in getting power from the generation site to where it’s consumed. One solution is to add hundreds or thousands of miles of transmission lines.
“That’s very costly and very time-consuming, and sometimes the cost is just passed down to residents in a utility bill increase,” said Shaolei Ren, associate professor of electrical and computer engineering at the University of California, Riverside.
One $5.2 billion effort to expand lines to an area of Virginia known as “data center alley” was met with opposition from local ratepayers who don’t want to see their bills increase to fund the project.
Another solution is to use predictive software to reduce failures at one of the grid’s weakest points: the transformer.
“All electricity generated must go through a transformer,” said VIE Technologies CEO Rahul Chaturvedi, adding that there are 60 million-80 million of them in the U.S.
The average transformer is also 38 years old, so they’re a common cause for power outages. Replacing them is expensive and slow. VIE makes a small sensor that attaches to transformers to predict failures and determine which ones can handle more load so it can be shifted away from those at risk of failure.
Chaturvedi said business has tripled since ChatGPT was released in 2022, and is poised to double or triple again next year.
VIE Technologies CEO Rahul Chaturvedi holds up a sensor on June 25, 2024, in San Diego. VIE installs these on aging transformers to help predict and reduce grid failures.
VIE Technologies
Cooling servers down
Generative AI data centers will also require 4.2 billion to 6.6 billion cubic meters of water withdrawal by 2027 to stay cool, according to Ren’s research. That’s more than the total annual water withdrawal of half of the U.K.
“Everybody is worried about AI being energy intensive. We can solve that when we get off our ass and stop being such idiots about nuclear, right? That’s solvable. Water is the fundamental limiting factor to what is coming in terms of AI,” said Tom Ferguson, managing partner at Burnt Island Ventures.
Ren’s research team found that every 10-50 ChatGPT prompts can burn through about what you’d find in a standard 16-ounce water bottle.
Much of that water is used for evaporative cooling, but Vantage’s Santa Clara data center has large air conditioning units that cool the building without any water withdrawal.
Another solution is using liquid for direct-to-chip cooling.
“For a lot of data centers, that requires an enormous amount of retrofit. In our case at Vantage, about six years ago, we deployed a design that would allow for us to tap into that cold water loop here on the data hall floor,” Vantage’s Tench said.
Companies like Apple, Samsung and Qualcomm have touted the benefits of on-device AI, keeping power-hungry queries off the cloud, and out of power-strapped data centers.
“We’ll have as much AI as those data centers will support. And it may be less than what people aspire to. But ultimately, there’s a lot of people working on finding ways to un-throttle some of those supply constraints,” Tench said.
