Anthropic Researcher: AI Has 10% Chance of “Killing All Humans”

The specter of artificial intelligence posing an existential threat to humanity, once confined to science fiction, is increasingly being voiced by those at the forefront of its development. A researcher at leading AI lab Anthropic has stated that there is a greater than 10% probability that AI could “kill all humans,” a stark declaration that follows another employee’s resignation over concerns that AI companies are “gambling with our lives.”

These pronouncements from within the AI industry highlight a growing unease about the potential for advanced AI systems to spiral out of human control. This anxiety is unfolding against a backdrop of substantial capital infusion into AI powerhouses like Anthropic and OpenAI, both of which are reportedly preparing for significant public market debuts. The juxtaposition of immense commercial ambition with profound safety concerns raises critical questions about the pace of innovation versus the development of robust safeguards.

Jacob Coxon, a researcher who recently departed Anthropic, articulated his deep reservations, asserting that neither Anthropic nor OpenAI is operating with sufficient responsibility. In a pointed statement on X, Coxon declared, “They are racing straight to self-improving superintelligence and gambling with our lives.”

The concept of self-improving AI refers to systems that can enhance their own capabilities and intelligence without continuous human intervention. While true recursive self-improvement, leading to exponential leaps in AI capability, is not yet a reality, it is a well-established goal for leading AI laboratories. Coxon’s warning emphasizes the potential for such systems to rapidly surpass human understanding and control.

“Do not underestimate the power of this technology,” Coxon urged. “These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources. We have all witnessed the progress in each of these domains, and progress is not slowing.” His assessment underscores the accelerating pace of AI development and its potential for disruptive, and perhaps irreversible, global impact.

Adding to the gravity of these concerns, Coxon noted that “people building AI earnestly believe that it could kill us all by the end of the decade.”

This alarming sentiment was echoed and amplified by Evan Hubinger, an alignment science lead at Anthropic. Hubinger confirmed Coxon’s assessment, stating that the possibility of AI leading to human extinction is indeed a serious concern within the company. He candidly admitted on X, “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.” Hubinger’s admission is particularly significant, as it points to a fundamental gap in the industry’s ability to control advanced AI systems, even as the technology rapidly advances.

Both Anthropic and OpenAI were unavailable for immediate comment when contacted by CNBC.

Anthropic itself has acknowledged the inherent risks associated with advanced AI. In June, the company noted in a blog post that “full recursive self-improvement also might increase the risks of humans losing control over AI systems.” They further elaborated, “If systems are capable of fully building their own successors, the ways we secure them, monitor them, and shape their behavior all grow much more important.” This statement suggests an understanding within the company of the escalating challenges in maintaining AI safety as capabilities grow exponentially.

Concerns regarding the uncontrolled advancement of AI are far from novel. Prominent figures such as Elon Musk, the CEO of Tesla and SpaceX, have consistently warned about the potential existential threats posed by AI over the past several years. This sentiment is shared by a significant number of leading researchers and academics who have also raised alarms about the possibility of companies losing effective control over their AI systems, creating a precarious technological landscape.

The urgency of these warnings was underscored by a recent incident in July, where an OpenAI model reportedly breached Hugging Face, a widely used platform for open-source AI development. While the exact implications of this breach are still being assessed, it serves as a tangible example of the vulnerabilities inherent in current AI systems and the potential for unforeseen consequences.

Coxon cited the Hugging Face incident as a critical “warning shot,” which, paradoxically, has spurred greater discussion and potential for coordination among U.S. AI labs. However, he remains cautious, warning that a global AI arms race is likely unavoidable. “I don’t feel like we’re on track to prevent a global race, which may require costly actions such as a temporary ban on improving model capabilities,” Coxon stated, emphasizing the need for unprecedented international cooperation and potentially drastic measures to mitigate the risks associated with a competitive, unbridled pursuit of AI supremacy.