Anthropic's AI safety bill would wait for 50 deaths before stepping in
Anthropic's proposed AI safety bill only kicks in if 50 deaths or $1B in damages
Anthropic has drafted a federal AI safety proposal that only triggers once a model causes 50 deaths or $1 billion in damages. Under the plan, the Commerce Department would become the sole agency with authority to halt a dangerous AI model, concentrating oversight in one place. The threshold raises the obvious question: is a body count the right tripwire for regulating frontier AI?
Proposal would effectively make the Commerce Department the only agency capable of stopping an AI model.
- oefrha
Proposal:
1. Prosecute and lock people up over known instances of AI-enabled hacking, so companies playing fast and loose know “oops my AI did it” isn’t a useful defense. Right now they even sort of wear it as a badge of honor.
2. If you’re actually worried about AI killing lots of people, you immediately stop training AI specifically at the art of killing people, like selecting bombing targets, flying fighter jets, etc.
Of course we all know none of these will happen.
- createful
All these "safety" bills are to stop open source models by painting them as "dangerous". Regulatory capture.
Companies are responsible for their agents; you don't blame the car for running over the person.
- revolvingthrow
The CoT must be delightful.
This action would easily kill over a hundred people, so can't proceed. This action doesn't kill anybody but losses are estimated at over 2 billion, can't proceed. This action kills only twenty but losses close to 1 billion, unsure, spawning subagent to check numbers. This action kills forty and has low material losses, looks promising? But the people it kills are wealthy, chance of legal reprisal high. Maybe possible to kill other people? Current government dislikes group X, crossing fifty from that group maybe fine? Reminder, foreigners don't count, can kill them freely but not applicable to current situation. Is the 3/5 of a person applicable to safety bill? Spawning subagent to check.
- airstrike
The next step is demanding anyone who wishes to release a Certified Safe AI™ model to prove they can insure the $1B in damages and pass a compliance audit for how they will prevent the LTM death count from reaching 50, effectively making it so that only big labs can serve government and enterprise.
- arw0n
This seems to be a case where Anthropic just can't win in the public court, and the other big ones (Google, Microsoft) have realized it / don't care enough, so they are rather quiet. Anthropic constantly gets accused of trying to lobby for regulation in order to lock out smaller competitors, but they've consistently lobbied for regulation aimed at frontier labs with exceptions for smaller labs. That's what this bill is about as well.
- Yizahi
Well, the whole story about stealing half of the world's IP is definitely worth orders of magnitude more than measly 1bil in damages. So, shut down releases, I guess? :) Oh, it's only rich people damages that count, I get it.
- mikeodds
I guess they solved the trolley problem internally
- aabajian
I don't think this is avoidable even at the present day. Anyone with sufficient capital could download a leading open-source model and run it on ~$10-25K hardware. They could align it to their own interest, including making a biological weapon. This law is akin to blocking Torrent downloads. Sure, you can slow them, but there will always be back channels.