Anthropic's Own AI Safety Bill: A High Bar for Halting Models

On October 16, 2026, Reuters, citing Politico, reported that Anthropic is privately lobbying lawmakers with a frontier AI safety bill it drafted itself; an Anthropic spokesperson confirmed to Reuters that the bill reflects proposals in the company's "Advanced AI Framework." The core provision is a single line: the government could only block a model's release or use when a "foreseeable, imminent" risk exceeds $1 billion in losses or would cause 50 or more casualties — limited to three scenarios: bioweapons, model escape, and cyberattacks. This is episode 4 of this site's "agent accountability" serial, and the most dramatic one: episode 3 was about "Congress wanting to put a leash on AI" (criminal liability, intent presumption); in episode 4, the AI company hands back a cage of its own design — the bars drawn high, the door actually wide.

What happened: a bill with a sky-high bar

First, the mechanics of Anthropic's version (details come from three sources cited by Politico; the bill text has not been made public — stating that upfront):

The halt threshold: the government's only condition for halting a model's release or use would be a "foreseeable, imminent" risk meeting either of these: projected losses exceeding $1 billion, or projected casualties of 50 or more — and the risk type limited to three categories: bioweapons, model escape (models going rogue), and cyberattacks. Outside those three categories, the government could not halt anything, no matter what happened.

The assessment mechanism: independent assessors registered with the Commerce Department would regularly assess each lab; labs would pay the assessment fees, but fees could not be tied to assessment outcomes (to prevent "pay to pass"). Assessors discovering an "imminent enumerated risk" would have to report it to Commerce within 72 hours.

Who decides: the Commerce Department could seek a halt order from a court, decided by a judge of the US District Court for the District of Columbia. But the bill would explicitly bar any other government department or official from pausing model development, deployment, or internal use — funneling the "halt power" into one extremely narrow channel.

Translated: the cage Anthropic designed has only three bars (bio, escape, cyber), and each stands as tall as $1 billion or 50 lives. Outside those three bars, the government's hand cannot reach in; even the hand that can reach in must first pass through a court.

The most interesting contrast: two "cages" side by side

Place episode 3's bipartisan congressional version next to Anthropic's own version, and the tension jumps out:

• Who is liable: the congressional version holds operators and developers criminally and civilly liable; Anthropic's version says nothing about developer liability — it's about "when the government may halt."
• The bar: the congressional version triggers on "knew the risk and didn't build guardrails"; Anthropic's version requires $1 billion in losses or 50 casualties before a halt.
• The intent question: the congressional version (the CLAIM Act) directly presumes AI has subjective intent — "no intent" is not a defense; Anthropic's version avoids the subject entirely.
• Government power: the congressional version expands it (all 50 state attorneys general can enforce); Anthropic's version shrinks it (one narrow Commerce Department channel, other departments barred).

On one side, "mishaps mean prison"; on the other, "don't touch me below $1 billion" — this is the head-on clash over "who writes the rules." Congress wants to write rules for AI; the AI company says "I already wrote them." And the most delicate backdrop: while handing over this bill, Anthropic is itself under FTC investigation (on September 30 the FTC confirmed it is investigating OpenAI, Anthropic, and other AI companies). The investigated party handing the investigators a memo on "how I should be regulated" — the timing alone is worth noting.

One caveat

This bill's text has not been made public; every detail comes from three anonymous sources cited by Politico, relayed via Reuters — the sourcing level must be stated clearly: this is "sources say," not "the bill text." Moreover, it is currently only a lobbying draft Anthropic is privately circulating to lawmakers; no lawmaker has formally introduced it as a bill, and it is very far from becoming law. Per our rules: this is "how an AI company wants to be regulated" news, not "how Congress will regulate" news. This site will follow up if a lawmaker picks it up for introduction or the text goes public.

Sources

• Reuters: Anthropic Quietly Pitches Its Own AI Safety Bill to Lawmakers, Politico Reports
• Previously on this site: \g<1>Bipartisan Bill: Criminal Liability Proposed for AI Agent Mishaps\g<2>
• Previously on this site: Anthropic's Claude Mishaps Go Public, White House Announces Mandatory AI Incident Reporting