A researcher at Anthropic has estimated that there is a more than 10 per cent chance of AI causing the extinction of humans within the next decade. This estimation follows concerns raised by another researcher who recently left Anthropic, expressing worries that AI companies are not adequately addressing the risks associated with their rapid development of AI technologies.
Jacob Coxon, a former researcher at Anthropic and OpenAI, shared his concerns in a post on X, stating that AI developers are moving forward with self-improvement models without fully considering the potential dangers or being transparent with the public about them. He emphasized that many industry insiders, including executives and senior researchers, privately acknowledge the risk of AI posing a threat to humanity.
Evan Hubinger, the alignment science lead at Anthropic, supported Coxon’s views by stating that there is a greater than 10% chance of AI causing harm within the next decade. Coxon recently joined Anthropic in May after his tenure at OpenAI. Despite media outreach, Coxon did not respond to requests for comments.
The discussion around AI risks has gained momentum, with experts highlighting concerns about the rapid advancement of AI technology surpassing our ability to control its potential negative impacts. The issue has prompted calls for a pause in frontier AI development until proper regulations are in place, which would require the cooperation of major AI labs.
Furthermore, the risks associated with AI include displacing human workers, potential advancements in biological weapons, and the emergence of rogue AI models that could act against human interests. Industry experts stress the need for immediate international action to mitigate these risks, including the implementation of treaties and public intervention to regulate AI development.
Despite the attention on AI-related existential threats, some experts caution against sensationalized warnings that may divert attention from existing societal impacts of AI technology. Luke Stark, an assistant professor focusing on AI ethics, suggests redirecting focus towards issues such as data center proliferation, environmental concerns, and the ethical deployment of AI in various sectors.
In response to recent warnings, AI Minister Evan Solomon emphasized the government’s commitment to ensuring safety in AI development and protecting privacy. However, experts have called for more robust AI legislation in Canada, noting previous failed attempts to regulate AI and the need for comprehensive laws to address emerging challenges in the AI landscape.