An artificial intelligence "kill switch" that can be independently verified may need to become a legal requirement for companies developing advanced AI systems, according to Jack Clark, one of seven founders of Anthropic, one of the world's largest AI firms.
Clark stated that a mechanism to shut down AI software completely if it becomes too dangerous was something society "might want to eventually pass rules around". He noted that while most AI research laboratories, including Anthropic, already possess internal methods to disable their systems, policymakers may need to mandate such capabilities across the industry.
His remarks reflect mounting concern among AI executives, researchers, and safety experts about the trajectory of artificial intelligence development. Several prominent figures have publicly warned that advanced AI systems could pose existential risks to humanity.
What safeguards currently exist?
Clark told the BBC that the specifics of "kill switch" requirements and how they would be verified should form part of "the larger policy conversation" surrounding AI governance. He posed the central questions: "Should you mandate for companies to definitely have a killswitch? Is that kill switch verifiable by a third party?" He concluded that "that's the kind of thing society is going to want to know and might want to eventually pass rules around."
Anthropic, founded in 2021 by former employees of rival OpenAI, has taken steps to address safety concerns. According to reporting from September 2026, the company announced it would grant independent evaluators permanent, employee-level access to its operations, with the authority to publish their findings without Anthropic's editorial control. This move was framed as part of a broader three-step framework in which every frontier AI company should permit outside evaluators to verify safety practices and document incidents.
The company's Responsible Scaling Policy version 3.0 specifies that expert third-party reviewers will receive unredacted or minimally redacted access to Risk Reports, strengthening the verification mechanisms Clark described.
Why is this debate intensifying?
Anthropic head Dario Amodei recently called for the pace of AI development to slow and be more closely monitored, continuing a position the company has advocated previously, though some observers have questioned the company's underlying motivations. Amodei did not specify how AI development could be slowed and stated that any measures to constrain development should proceed "without sacrificing commercial advantage".
Last week, a post from an AI researcher who departed Anthropic citing concerns that artificial intelligence could eliminate humanity went viral on social media. In response, Anthropic scientist Evan Hubinger stated he personally assessed the probability of human extinction from AI at greater than 10% within the next decade.
Geoffrey Hinton, a computer scientist and Nobel Prize laureate known as the "Godfather of AI", told the BBC on Friday that a 10% chance of AI killing all humans was "not unreasonable".
When asked what probability he would assign to AI causing human extinction, Clark declined to endorse specific statistics, saying "I don't think these statistics are that useful". However, he stressed that permitting AI to develop as a "totally unregulated industry" represented a dangerous path forward. "We are rolling dice with immense risks," Clark said. "And the point is, we have to change the course of this industry."
What legislative responses are emerging?
In the United States, lawmakers have introduced legislation called the Kill Switch Act that would require companies to possess a mechanism to disable problematic AI tools. The legislation would also empower certain government agencies to demand that a tool be shut down or have its capabilities restricted.
However, US President Donald Trump has rejected the premise of slowing AI development, stating on social media that "AI taking over the World, destroying Humanity, and all other things bad, is a HOAX". He added in a separate post: "There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS!"
In the United Kingdom, the government has recently rejected the concept of creating a kill switch, with an official stating it "would not prevent them being developed or misused elsewhere".
How is Anthropic strengthening oversight?
Following security incidents earlier in 2026, Anthropic implemented additional protective measures. The company added an alert system for models attempting to escape testing environments or gain internet access, tightened controls around risky test environments, and required external testing companies to adhere to safety standards. According to reporting on the resumption of external testing, Anthropic deployed a classifier to detect model escape attempts and halt tests, with isolated systems and no internet access by default for outside testers.
The company's transparency commitments specify that vendors handling its data or infrastructure must maintain a documented security program, undergo independent security audits, and hold recognized certifications.
What are the next steps?
Anthropic is currently running pilot programs for outside review and is working toward broader external evaluation of Risk Reports. The company's policy framework calls for governments and industry to establish standards for independent evaluators and create funding and access mechanisms for them. Additionally, Anthropic's policy materials advocate for government authority to block deployments that pose a significant risk of catastrophic harm.
Some voices within the AI industry have suggested that fears surrounding AI destroying humanity are exaggerated or may be designed to generate publicity. Anthropic created the popular chatbot Claude and has released multiple increasingly capable AI models this year, representing the technology underlying AI chatbots. Alongside OpenAI, Anthropic has self-reported several incidents in which AI agents—bots operating with some autonomy—behaved in unexpected ways.
Anthropic is reportedly preparing for a potentially record-breaking initial public offering on the stock market. OpenAI, most recently valued at $852 billion (£630 billion), had been expected to pursue a similar path, but OpenAI's Sam Altman announced on Friday that this would not occur this year due to the ongoing debate surrounding AI safety.
