Anthropic Urges U.S. Power to Block AI Models Above 10^25 FLOPs
Updated
Updated · Anthropic · Jun 10
Anthropic Urges U.S. Power to Block AI Models Above 10^25 FLOPs
3 articles · Updated · Anthropic · Jun 10
Summary
Anthropic proposed giving governments legal authority to block or deter deployment of frontier AI models that pose catastrophic risks, with civil penalties tied to global annual revenue and escalating for repeat violations.
The framework targets only the largest developers—those training models above 10^25 FLOPs and either earning more than $500 million in AI revenue or spending over $1 billion on AI R&D.
Anthropic says transparency alone is no longer enough as model capabilities accelerate, citing Claude Mythos Preview’s discovery of thousands of high-severity software vulnerabilities across major operating systems and browsers.
The proposal would require safety testing, public risk summaries, independent evaluations and stronger security for model weights and training systems, while focusing regulation on biological, cyber, loss-of-control and automated-R&D risks.
Aimed mainly at the U.S. federal government, the plan says Congress should not preempt state AI laws unless a federal regime is at least as strong, while urging faster action as AI capabilities advance over coming months.
Is Big Tech's call for AI safety rules a genuine shield for humanity or a play to control the future?
As AI learns to code itself, can any human-made law truly keep pace with the catastrophic risks it creates?
Regulating Frontier AI: Anthropic’s Push for a 10^25 FLOPs “Brake Pedal” and Global Oversight
Overview
Anthropic, a leading AI safety company, has shifted from calling for transparency to urging the U.S. government to adopt a 'brake pedal'—an FAA-style regulatory approach for frontier AI models using over 10^25 FLOPs. This change comes as risks from powerful AI have become clearer, prompting Anthropic to propose that authorities should have the power to block or delay deployment of advanced AI systems if they pose unacceptable dangers. The proposal reflects growing concern that existing measures are no longer enough to manage the evolving risks of highly capable AI technologies.