Anthropic, the company behind Claude, has confirmed it is quietly sharing its own AI safety bill with members of Congress. The Anthropic AI safety bill would let the federal government stop a powerful model from being released or used, but only when a judge agrees the risk is extreme. It is a rare case of an AI lab writing the rules it wants to live under, and it lands while Washington still has no federal AI law at all.
What the Anthropic AI Safety Bill Would Do
The draft sets a very high bar before the government can step in. A model could be blocked only if it poses an imminent risk of killing or seriously injuring more than 50 people, or of causing more than $1 billion in damage. Even then, the danger has to come from one of three specific sources.
- The creation or release of a biological weapon.
- A model slipping out of its developer’s control.
- A cyberattack.
The details first came out in a Politico report on Friday, October 16, and Anthropic confirmed the draft the same day. A company spokesperson said the bill gives the government the ability to “block or deter dangerous deployments of the most powerful models.”
Testing, Disclosure and Security Rules
Beyond the emergency brake, the bill asks frontier labs to do more routine work. Developers would have to test their most powerful models, share what they find with the public, submit to independent evaluation and keep a strong security program in place. Anthropic says the draft turns ideas from its Advanced AI Framework, released earlier this year, into legal language.
How Independent AI Evaluators Would Work
A big part of the plan relies on outside reviewers. These evaluators would register with the Commerce Department and check each lab and its frontier models on a regular schedule. Labs could pay them, as long as the payment does not depend on what the evaluation finds.
If an evaluator decides a model carries an imminent risk from the list above, it must report that to Commerce within 72 hours of finishing its review. Commerce could then ask a federal judge in the District of Columbia to order a suspension. That echoes the outside-audit approach Anthropic has already tried, such as giving METR deep access to review its systems.
| Feature | What the draft says |
|---|---|
| Harm threshold | Over 50 people killed or seriously hurt, or over $1 billion in damage |
| Covered risks | Bioweapons, loss of control, cyberattacks |
| Who can halt a model | A federal judge in D.C., at Commerce’s request |
| Evaluator reporting window | 72 hours after finishing an assessment |
| Other agencies | Barred from suspending a model on their own |
| State laws | Would override some, including California’s |
Where Critics Say the Safety Bill Falls Short
The bill cuts both ways. It creates a path to stop a dangerous model, but it also says no other federal department, agency or officer can suspend a model’s development, deployment or internal use. In practice, that funnels every shutdown through one agency and one court.
It would also override certain state AI laws, such as California’s. That is a shift from what Anthropic wrote in June 2026, when it said states should keep full authority to legislate in the absence of a strong federal law. California has kept moving on its own, with Governor Newsom recently ordering a study of AI kill switches.
Charlie Bullock of the Institute for Law and AI called the draft pretty reasonable and said it goes further than other frontier labs have gone. He also said he sees it as weaker than the FRONTIER Act, a competing bill introduced in the House this summer.
Where the Bill Stands in Congress
For now, this is a company proposal, not a bill anyone has formally introduced. Anthropic would not say how widely it has shared the draft or how lawmakers have responded, though it has pushed these ideas with federal and state officials since June. Congress is out of session until after next month’s midterm elections, so nothing will move quickly.
The timing is notable. The White House has mostly leaned on voluntary pledges, while a new federal task force is only starting its review. Regulators have also grown more active, with the FTC recently sending safety subpoenas to major labs.
What This Means for You
You will not see any change in how Claude or other chatbots work today. Still, this debate shapes how much protection you get as AI tools grow more capable. Here is what to keep in mind:
- The proposed threshold is very high, so everyday harms like bad advice or biased answers are not covered by the shutdown power.
- Public testing reports could give you more insight into how safe the models you use really are.
- If a federal law overrides state rules, protections in places like California could get weaker or stronger depending on the final text.
- Watch for this draft, or pieces of it, to show up after the midterms when Congress returns.
The takeaway: an AI lab is offering lawmakers a narrow emergency brake, and the fight now is over whether narrow is good enough.
Anthropic’s leaders have been warning that oversight is falling behind, as co-founder Jack Clark argued in his recent regulation call. You can follow more updates like this on our AI news page.
Sources
This article is based on reporting from Law Commentary, The Hindu and the Free Press Journal.
Conclusion
Anthropic’s draft bill offers Congress a ready-made framework for testing frontier models and stopping the worst of them. Its supporters see a sensible first step, while critics worry about the high bar, the single path to a shutdown and the override of state laws. Nothing will pass before the midterms, but the draft gives lawmakers a concrete starting point. It also puts pressure on other AI labs to say what rules they would accept. How Congress responds will say a lot about who ends up writing America’s AI safety rules.
