Is Microsoft’s $100 billion ‘Stargate’ OpenAI supercomputer AI’s ‘Star Wars’ moment?

To put this in context, Microsoft is known to have spent more than “several hundred million dollars” to build the clusters used to train OpenAI’s current top-of-the-line model GPT-4, which OpenAI CEO Sam Altman also has said cost more than $100 million to train. It’s also known that OpenAI is already training a successor model to GPT-4, likely called GPT-5, on one of Microsoft’s existing data centers. And we know that Microsoft last year broke ground on a new $1 billion data center in Wisconsin that analysts believe is intended to house the chips for training OpenAI’s next-generation models—probably due out in 2025 or 2026. The Information reported that this supercomputing cluster in Wisconsin may eventually cost as much as $10 billion, once the price of specialized Nvidia chips used for AI applications are factored in. So Stargate is anywhere from 10 to 100 times more expensive than any of the data centers Microsoft currently has on the books.

Of course, $100 billion is a lot of money, even for Microsoft. It’s more than three times what the company spent on capital expenditures in 2023 and twice the amount it is on pace to spend this year. It is also more than almost any company, including those in far more capital-intensive sectors, spends on capex annually: Saudi Aramco, for instance, spent about $50 billion on capital projects last year. It’s also two-thirds of the entire amount Amazon’s AWS has said it plans to spend on all new data centers over the next 15 years.

Skeptics of today’s approaches to AI seized on the Stargate story as an “AI jumps the shark” moment. The only way a single OpenAI model could justify such an outlandish investment on a single data center, they said, was if that model were in fact AGI—or artificial general intelligence, or a single AI system that could perform most cognitive tasks as well or better a human. Achieving AGI is OpenAI’s founding mission. But since some of these skeptics doubt AGI is achievable within the next decade, they predicted this investment—if Microsoft does, in fact, make it—would prove foolish. Gary Marcus, the AI expert who has emerged as a perpetual critic of most neural network-based approaches to AI, called Stargate “the second worst AI investment in history”—comparing it to the more than $100 billion companies have plowed into self-driving cars, which today only operate in a few limited geographies. But remember, that $100 billion was spent by many different investors and companies. Stargate would be all Microsoft.

What’s more, Microsoft has to hope that whatever Stargate is being used for, it is not in fact to train AGI, since Microsoft’s partnership with OpenAI only entitles the tech giant to commercialize OpenAI’s technology that falls short of AGI. Once OpenAI’s board decides AGI has been achieved, the company doesn’t have to share that technology with Microsoft. For Microsoft to invest so much in training an OpenAI model, it must be reasonably sure that the model will be very capable, but not yet so capable as to qualify as AGI.

Beyond this inherent contradiction at the heart of the Microsoft-OpenAI partnership, Project Stargate signals several other things. Jack Clark, who in addition to heading policy at OpenAI rival Anthropic writes a good newsletter on AI, Import AI, points out that the seemingly soaring capital intensity of advanced AI has important implications for AI policy. Capital-intensive industries, such as mining or oil and gas, tend to be highly concentrated, with a handful of large companies dominating the market. They also tend to be heavily regulated. Project Stargate may be an indication this is where AI is headed.

But there are other interesting implications for both policy and corporate strategy. In some ways, we may come to see Stargate as AI’s Star Wars moment. And no, I don’t mean the movie. I am referring to the nickname for the Strategic Defense Initiative, the space-based antiballistic missile system President Ronald Reagan announced the U.S. Intended to build in 1983. The important thing about SDI was not the tech—most of which didn’t exist when Reagan declared the U.S. Would build the system (and in fact, the whole project was later scrapped). What mattered was the audacious ambition and its eye-watering projected price tag—which at the time was estimated to be $30 billion ($93 billion in today’s dollars) over its first six years with estimates ranging from $100 billion to $1 trillion over the first decade. Many analysts have since claimed that those costs convinced the Soviet Union’s leadership that the country could not compete economically with the U.S. The Soviets simply didn’t have the money to try to match the American investment while also modernizing their missiles to try to defeat the U.S. Antimissile system. There are indications that this realization was among the factors that persuaded the Soviet leadership to begin economic reforms and political liberalization that ultimately hastened the collapse of the entire Soviet system.

Stargate may play a similar role between both companies—and countries. Microsoft currently operates with about a 35% profit margin. Meta’s is similar. Google’s is slightly less at 24%. Amazon, whose AWS service is another big cloud rival, operates at just 8% margins. If you are Amazon, are you really willing to spend $100 billion on a single data center to match Microsoft? What happens when the costs for the data center for the model beyond this next generation hits $200 billion? Stargate could well convince one of these companies that it can’t afford to be in the frontier model business.

The geopolitics of this are interesting too. Most of the large companies building frontier AI models are American. If you are China, which sees advanced AI, and certainly AGI, as a strategic asset, you are facing the prospect of the U.S. Having not one but perhaps four or five $100 billion AI supercomputers being built in the next five years, all at no direct cost to the U.S. Government. Meanwhile, China has been subsidizing some of the efforts by its internet companies, such as Baidu, to build AI supercomputing clusters, partly because these companies have been cut off from the most advanced AI chips produced by Nvidia and partly because of fears they may be falling behind their U.S. Tech company rivals. At some point, Beijing may decide it can’t afford to match the American effort like-for-like. This could have several effects, one of which might be to drive China to find other ways to race for AGI—maybe trying to find a path to AGI that does not depend on transformer-based neural networks and GPUs. It could put more money into different kinds of algorithms or different kinds of chips, perhaps achieving a surprise breakthrough. It could also redouble efforts to steal and then replicate U.S.-made AI models.

We’ll be watching to see how these dynamics play out in the coming months and years.

There’s more AI news below. But first, if you’re enjoying reading this newsletter, how would you like to participate in a live version—chatting in person with me and many of the world’s foremost experts on deploying AI within companies? If that sounds intriguing, THIS IS YOUR LAST CHANCE to apply to attend the Coins2Day Brainstorm AI conference in London on April 15-16. I’ll be there cochairing the event and moderating sessions. You will get to hear from Google DeepMind’s Zoubin Ghahramani, Microsoft chief scientist Jaime Teevan, Salesforce chief ethical and human use officer Paula Goldman, as well as Shez Partovi, the chief innovation and strategy officer for Royal Philips, Accenture’s chief AI officer Lan Guan, Builder.ai CEO Sachin Dev Duggal, and many others. Email [email protected] to apply to attend. I hope to see you there!   

With that, here’s the AI news.

Jeremy Kahn
[email protected]
@jeremyakahn

AI IN THE NEWS

OpenAI unveils voice AI system. OpenAI has revealed it has trained and tested a voice cloning AI called VoiceEngine that requires only a 15-second voice recording to generate natural-sounding synthetic audio that is almost indistinguishable from the original. The company said in a blog post that it developed the model in late 2022 and had been testing it in a limited release with a select group of developers. The model also powers OpenAI’s existing text-to-voice API and the voice users get when asking ChatGPT to voice its responses. Developers have used the model to help translate audio content into new languages and to help those who have medical conditions that have resulted in the loss of their voices. But the company warned that it was holding back on a wider release of the model as it encouraged a broader discussion around risks. OpenAI said it believes companies should cease to rely on voice-based authentication for securing bank accounts and other sensitive data and that there needs to be more work on digital watermarks and technology to track the provenance of audio content. You can read more in this Bloomberg story.

OpenAI makes ChatGPT available without an account. The company announced Monday it has made its popular chatbot available to users for the first time without requiring them to create an account and login. Previously, users were required to register before using the free version of ChatGPT. The company said those who do sign up gain access to more features, including the ability to save chat histories and interact with the AI chatbot using voice instructions. It also said it was instituting new safeguards for the instant-access version to prevent people from using malicious prompts to try to get the model to generate responses that might be harmful or dangerous. Although OpenAI said ChatGPT now has 100 million weekly users, its decision to provide instant access without signups for the chatbot may be an effort to combat slowing user growth. Data from SimilarWeb, which tracks traffic to websites, has shown declining traffic to OpenAI’s website in a number of months, with overall traffic down 11% from its May 2023 peak through February, and with relatively modest growth for its mobile app, according to a recent story in tech newsletter The Wrap.

Perplexity AI to begin selling ads. The generative AI search engine, which has gained a cult following among many techies and that hopes to eventually knock Google off its perch as the dominant search site, plans to start selling advertising, according to a story in AdWeek. The company will let brands pay to help shape the “related questions” that are suggested to users after they perform a search and get an initial AI-generated response. Those related queries account for about 40% of all Perplexity’s searches, the company said. Perplexity, which was founded by Meta and OpenAI alumni and has garnered some $100 million in venture capital funding so far, has won over users by providing responses that seemed less influenced by search engine optimization and advertising than many Google searches, and because the generative AI responses provided footnoted corroboration for its statements.

U.S. And U.K. Sign agreement to jointly develop safety tests for AI models. The memorandum of understanding, which was signed by U.S. Commerce Secretary Gina Raimondo and British Technology Secretary Michelle Donelan on Tuesday, follows on from commitments the two countries made at last November’s AI Safety Summit in Britain. The tests will apply to the most advanced AI models companies are developing. The U.S. And U.K. Have both created AI Safety Institutes to scrutinize these “frontier” AI models and have said they will conduct at least one joint test of a publicly available model in the coming year. You can read more about the new British-American agreement here.

EYE ON AI RESEARCH

A SAFE-er way to avoid LLM hallucinations? Google DeepMind researchers have published research on a way to reduce AI hallucinations. Called SAFE, for Search-Augmented Factuality Evaluator, the method involves taking the response generated by a large language model and then using either the same large language model or a different one to break that response into individual statements containing facts. Each of these is then sent to a search engine, and the evaluator LLM is asked to determine if the information in the search results supports or contradicts the statement. The LLM is asked to explain its reasoning for its conclusions. The researchers found that on a set of 16,000 individual facts, SAFE agreed with the conclusions of human annotators 72% of the time. When the researchers looked at a random sampling of 100 cases where the humans and SAFE disagreed, they concluded SAFE was actually correct 76% of the time. It also said SAFE was 20 times cheaper than hiring human fact-checkers. They evaluated a number of models using SAFE and found that GPT-4 Turbo, the latest model OpenAI uses to power the paid version of ChatGPT, performed best, with a 92% accuracy rate, with Anthropic’s Claude 3.0 at 88% and Google’s own Gemini Ultra at 86%. You can read the paper on the non-peer-reviewed research repository arxiv.org here.

FORTUNE ON AI

As OpenAI targets Hollywood with Sora, Runway’s CEO is waiting for the $86 billion Goliath with a ‘sling and a stone’ —by Kylie Robison

Why Amazon’s multibillion-dollar AI alliance with Anthropic isn’t the game-changer it needs to remain king of the cloud —by Sharon Goldman

OpenAI tipped to become the world’s first trillion-dollar privately held startup by former Google China president —by Christiaan Hetzner

Inside the $1 billion love affair between Stability AI’s ‘complicated’ founder and tech investors Coatue and Lightspeed—and how it turned bitter within months —by Jessica Mathews and Allie Garfinkle

AI CALENDAR

April 15-16: Coins2Day Brainstorm AI London (Register here.)

May 7-11:  International Conference on Learning Representations (ICLR) in Vienna

May 21-23: Microsoft Build in Seattle

June 5:  FedScoop’s FedTalks 2024 in Washington, D.C.

June 25-27:  2024 IEEE Conference on Artificial Intelligence in Singapore

July 15-17: Coins2Day Brainstorm Tech in Park City, Utah (register here)

Aug. 12-14: Ai4 2024 in Las Vegas

BRAIN FOOD

Has the EU cast too wide a net when it comes to policing AI safety? That is a question that Anthropic policy honcho Jack Clark, getting his second hat tip of the week, recently pondered. He noted that the EU AI Act says that any general-purpose AI model that represents more than 10^25 FLOPS (or floating point operations per second) of computing power is automatically considered to represent a potential “systemic risk” and needs to “to assess and mitigate risks, report serious incidents, conduct state-of-the-art tests and model evaluations, ensure cybersecurity and provide information on the energy consumption of their models.” Meanwhile, in the U.S., the Biden Administration’s Executive Order on AI mandated that any person or company training a model at more than 10^26 FLOPS needed to report it to the government and show the work they were doing to test the model and ensure its safety.

Clark did some back-of-the-envelope calculations and estimated that a 10^25 FLOPS model costs about $7 million to train on the latest GPU hardware, while a 10^26 FLOPS model costs about $70 million to train. Clark rightly points out that very few companies outside those already identified as being at the cutting edge of AI—OpenAI, Anthropic, Google, Microsoft, Meta, Nvidia, and maybe a few others such as Cohere—would contemplate a training run as expensive as $70 million. But lots of smaller venture-backed startups and lots of big companies could probably afford $7 million. So the question Clark asks is whether the EU’s new AI Safety Office has bitten off more than it can chew and will find itself having to regulate far more companies and models than it anticipated. If you want to see Clark’s calculations you can read more here.