Anthropic, the AI company behind the Claude chatbot, has an internal researcher who believes artificial intelligence could lead to human extinction within the next ten years. Evan Hubinger, Anthropic's lead for AI alignment science, stated that he "earnestly" believes this possibility exists and estimates the probability at over 10% within the next decade. This sentiment was shared in response to the resignation of AI researcher Jacob Coxon, who cited safety concerns as his reason for leaving Anthropic. Hubinger admitted that the company currently lacks a definitive plan to ensure the safety of superintelligent AI and is not demonstrably on a path to achieve it.
While Anthropic's latest risk report indicates a low risk of extinction from current AI models, Hubinger expressed worry about the potential for superintelligence to emerge through recursive self-improvement—a process where AI continuously rewrites and enhances its own code. Coxon echoed these concerns, asserting that leading AI companies like Anthropic and OpenAI are not acting with sufficient responsibility regarding AI safety. He warned that these systems are rapidly evolving into superhuman entities capable of hacking, revolutionizing industries, and acquiring significant power and resources, with progress showing no signs of slowing.
Coxon suggested that Anthropic is driven by a competitive race to develop advanced AI first, even amid these safety concerns. He also noted that while OpenAI might not be taking the issue seriously enough, Anthropic recognizes the "civilizational stakes" and feels a responsibility to act, even if it means proceeding despite the risks. These statements have ignited considerable debate within the tech community and beyond.
Senator Bernie Sanders has voiced his concerns about AI's potential threat to the economy, privacy, democracy, and public well-being, calling for congressional action against major tech companies. Conversely, technology reporter Taylor Lorenz described Hubinger's comments as "sanctimonious doomer" statements that could lead to detrimental legislation, urging him to provide concrete evidence rather than vague warnings that might result in "terrible policy."
Adding another layer to the discussion are Anthropic's expected public offering this year, with a valuation potentially exceeding $2 trillion. Some speculate that the company's public pronouncements on existential risk could be a strategic maneuver to generate market fear, positioning Anthropic to offer its safety solutions as a remedy. Similarly, previous statements by Anthropic CEO Dario Amodei about AI potentially eliminating half of all entry-level white-collar jobs have been interpreted by some as fundraising tactics.
The intersection of AI and the video game industry is also a point of contention. EA Sports recently revealed its use of generative AI to replicate a commentator's voice, a move that mirrors broader industry debates surrounding AI's impact, including fears of job displacement and increased workloads.