Microsoft Azure CTO: US data centers will soon hit size limits
The data centers that make generative AI products like ChatGPT possible will soon reach size limits, according to Microsoft Azure Chief Technology Officer Mark Russinovich, necessitating a new method of connecting multiple data centers together for future generations of the technology.
The most advanced AI models today need to be trained inside a single building where tens (and soon hundreds) of thousands of AI processors, such as Nvidia’s H100s, can be connected so they act as one computer.
But as Microsoft and its rivals compete to build the world’s most powerful AI models, several factors, including America’s aging energy grid, will create a de facto cap on the size of a single data center, which soon could consume multiple gigawatts of power, equivalent to hundreds of thousands of homes.
Already, some parts of the national grid are overwhelmed on hot days, when air conditioners are running full blast, forcing rolling blackouts and brownouts.
Microsoft has been working furiously to help add capacity to the grid, inking a deal to reopen the Three Mile Island Nuclear power plant, launching a $30 billion fund for AI infrastructure with BlackRock and inking a $10 billion deal with Brookfield for green energy, among other projects.
Overhauling the US’ energy infrastructure was a big part of the 2022 Inflation Reduction Act, which provided $3 billion in incentives for building out transmission lines, among other priorities. But companies like Microsoft can’t afford to wait around for more money from Washington, on top of the time it would take to deploy those funds.
Google will back the construction of seven small nuclear-power reactors in the U.S., a first-of-its-kind deal that aims to help feed the tech company’s growing appetite for electricity to power AI and jump-start a U.S. nuclear revival.
Under the deal’s terms, Google committed to buying power generated by seven reactors to be built by nuclear-energy startup Kairos Power. The agreement targets adding 500 megawatts of nuclear power starting at the end of the decade, the companies said Monday.
The arrangement is the first that would underpin the commercial construction in the U.S. of small modular nuclear reactors. Many say the technology is the future of the domestic nuclear-power industry, potentially enabling faster and less costly construction by building smaller reactors instead of behemoth bespoke plants.
“The end goal here is 24/7, carbon-free energy,” said Michael Terrell, senior director for energy and climate at Alphabet’s Google. “We feel like in order to meet goals around round-the-clock clean energy, you’re going to need to have technologies that complement wind and solar and lithium-ion storage.”