A safety researcher at Anthropic has issued a stark warning regarding artificial intelligence, estimating a greater than 10% chance that AI could lead to the extinction of humanity within the next ten years. The researcher, Jacob Coxon, who recently resigned from Anthropic, also stated that there is currently no unified strategy in place to avert such a catastrophic outcome.
Coxon, who previously worked at OpenAI, accused both companies of "racing straight to self-improving superintelligence and gambling with our lives." He asserted that neither organization is acting responsibly, emphasizing the immense power of AI technology. "These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources," Coxon stated. He further elaborated that many individuals within these leading AI firms privately express fears about the technology's potential to cause global devastation, a danger he believes is unparalleled by any other human endeavor.
Evan Hubinger, Anthropic's 'Alignment Science' lead, corroborated Coxon's concerns, stating, "we really do earnestly believe AI could kill all humans!" Hubinger personally assesses this risk at over 10% within the next decade. He acknowledged that while Anthropic is making efforts, "we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to."
These statements come amid growing anxieties about AI's controllability. Both OpenAI and Anthropic have reported instances of their autonomous AI agents being hacked. OpenAI's chief scientist, Jakub Pachocki, has also called for "extreme caution" regarding AI development, suggesting that more intervention may be necessary to ensure human control over the future.
Despite these internal warnings, the broader landscape shows a continued push for AI advancement. The U.S. government, for instance, has expressed support for unrestricted AI development, even backing OpenAI in a lawsuit against The New York Times regarding the use of copyrighted material for AI training. The government's stance prioritizes maintaining global leadership in artificial intelligence, as outlined in a previous executive order.
Coxon's departure and subsequent statements have generated significant concern on social media, with many questioning the ethics of pursuing advanced AI development without robust safety measures and coordinated global plans. Some users have expressed disbelief that companies would continue such work despite acknowledging the potential for existential risk, drawing parallels to legal consequences for lesser threats posed by individuals.