OpenAI's AGI chase is tricky concept and contract

NEW YORK, March 16 (Reuters Breakingviews) - Imagine a wildly successful partnership that dissolves once something happens. Except it’s not clear what this event is, whether it’s occurred, or even if it’s possible. That’s the challenge facing OpenAI and Microsoft (MSFT.O), opens new tab as they contemplate the possible arrival of artificial general intelligence. The opacity of their agreement, and the huge sums of money at stake, make it almost inevitable that a theoretical debate ​will turn into a contractual dispute.

The concept known as AGI is the point where autonomous systems can match or surpass humans at intellectual tasks. In Silicon Valley this is both an aspirational goal and existential ‌threat. Creating a self-learning system could, theoretically, result in outcomes ranging from a golden age for humanity to extinction.

The Reuters Iran Briefing newsletter keeps you informed with the latest developments and analysis of the Iran war. Sign up here.

OpenAI was founded in 2015 with this explicit objective. Sam Altman, Ilya Sutskever, Elon Musk and others set out to create AGI, which they defined, opens new tab as “highly autonomous systems that outperform humans at most economically valuable work, for the benefit of all humanity”. In line with the idea that the love of money is the root of evil, OpenAI was founded as a non-profit.

Ego and goodwill were not sufficient, though: the AI race also required huge financial resources. So the company established a for-profit arm ​in 2019, and a partnership with Microsoft. The software giant invested $1 billion and provided computing infrastructure in exchange for access to OpenAI’s technology. The startup insisted that, once AGI was achieved, exclusive rights for the technology would revert ​to the startup.

This was probably an easy concession to make at the time. Yet as AI systems have rapidly grown more capable, defining AGI has become a less abstract question. On ⁠measures such as visual reasoning, English-language understanding, or competition-level math, advanced systems have surpassed average human benchmarks, according to Stanford Institute for Human-Centered AI.

Here’s where the conceptual debate turns into a contractual one. Like Supreme Court Justice Potter Stewart’s subjective explanation of obscenity — “I ​know it when I see it” — AGI was poorly defined. OpenAI’s clause was neither concrete nor easily observable, as researchers at Google pointed out, opens new tab in a 2023 paper which tried to lay out a framework for such declarations.

Take the provision that the milestone would ​be reached when systems could outperform most humans at economically valuable work. What does “most people” mean? Can you measure this without deployment in the real world, or if use is slowed for legal or ethical reasons? Finally, the economic value of many jobs is hard to define.

Meanwhile, training and deploying AI models requires voracious amounts of cash. Microsoft, Alphabet (GOOGL.O), opens new tab, Meta Platforms (META.O), opens new tab and Oracle (ORCL.N), opens new tab have sharply increased capital expenditure in response to these needs and plan to spend over $700 billion this year. As a startup without existing revenue sources, OpenAI needed support.

Microsoft ended up pumping in a total of $13 billion. ​That wasn’t enough for OpenAI, though, and the increasingly complex agreement between the two companies, several parts of which depended on the AGI clause, chafed with both sides.

Other industries have learned that vague agreements and large sums of money don’t mix. Take ​catastrophe bonds, which help companies lay off the financial risk from hurricanes, floods or pandemics. These events are rare, but the $1.4 trillion bill from a storm hitting a big U.S. city like Miami might make the whole insurance industry teeter. So companies bring in outsiders willing to ‌take on gigantic ⁠but low-probability risks in exchange for payments. Initially opaque agreements led to bitter disputes, such as the five-year legal fight between a reinsurer and bondholders over what exactly was covered following a 2008 hurricane.

Catastrophe bonds have since moved towards more concrete and easily observable triggers, lowering uncertainty and avoiding disputes. Think of measuring wind speeds at a specific location, say, rather than overall insurance industry losses or whether a government declares a state of emergency.

In biotech mergers, companies use contingent payments to bridge the gap between buyer and seller when valuing experimental drugs. These usually incorporate non-debatable triggers, like whether the U.S. Food and Drug Administration has approved the treatment by a certain date.

OpenAI and Microsoft have since reached a form of détente, probably because extended negotiations might end up with ​both sides losing out. A new agreement, opens new tab, signed late last year, says ​an independent panel must verify any OpenAI claim that ⁠AGI has been achieved.

Microsoft keeps a 27% stake in the company, most recently valued at $840 billion, and has the right to some technology until 2032, even if AGI is achieved. Sam Altman’s company now can pursue partnerships or an initial public offering. There is still reason for tension, though. OpenAI sends about 20% of its revenue to Microsoft, and that essentially goes away if AGI ​occurs. Such a large financial shift would make a big difference to OpenAI’s finances as it prepares to sell stock to the public.

The argument over how to define AGI hasn’t ​really been settled. Sam Altman said ⁠late last year in a podcast, opens new tab that people didn’t really notice and it didn’t change the world much, but “we built AGI”. Microsoft CEO Satya Nadella appears to focus more on large effects, claiming last year that AGI would be “senseless benchmark hacking, opens new tab” and the real benchmark of success would be the world economy growing at an improbable 10%.

The debate has also moved on. Even AI evangelists like Altman admit AI still does some things poorly, such as complex reasoning and the ability to learn. He now touts the idea of superintelligence and AI systems ⁠doing things humans ​cannot do, or systems doing jobs, like being U.S. president, better than a human. That’s still a woolly definition, though.

Altman is still chasing. OpenAI’s recent $110 ​billion fundraising included a big contribution from Amazon (AMZN.O), opens new tab which could hang on whether the company goes public or achieves AGI, the Information reported, opens new tab.

In an interview with CNBC following the agreement, Altman said AI progress remains rapid, and since AGI is a “near-term thing”, the company isn’t doing new deals that stop when it’s reached. That ​timeline may be optimistic, but if so, the dispute still has time, and incentive, to flare up again.

Follow Robert Cyran on Bluesky, opens new tab.

For more insights like these, click here, opens new tab to try Breakingviews for free.

Editing by Peter Thal Larsen; Production by Maya Nandhini

• Suggested Topics:
• Breakingviews

Breakingviews
Reuters Breakingviews is the world’s leading source of agenda-setting financial insight. As the Reuters brand for financial commentary, we dissect the big business and economic stories as they break around the world every day. A global team of about 30 correspondents in New York, London, Hong Kong and other major cities provides expert analysis in real time.

Sign up for a free trial of our full service at and follow us on X @Breakingviews and at www.breakingviews.com. All opinions expressed are those of the authors.

Share

• X

• Facebook

• Linkedin

• Email

• Link

Purchase Licensing Rights

Robert Cyran

Thomson Reuters

Robert Cyran, U.S. tech columnist, joined Breakingviews in London in 2003 and moved four years later to New York, where he continues to cover global technology, pharmaceuticals and special situations. Robert began his career at Forbes magazine, where he assisted in the startup of the international version of the magazine. Before working at Breakingviews he worked as a market researcher and reporter covering the pharmaceutical industry. Robert has a Masters degree in economics from Birmingham University and an undergraduate degree from George Washington University.

• Email