An open letter to Dario Amodei: if you mean it, open the weights
An open letter to Dario: if you mean it, open the weights
Jacob Gold challenges Anthropic CEO Dario Amodei to back his call for slowing frontier AI with a law requiring every publicly released model to be open weights. Gold argues that all regulations Amodei has proposed so far end in regulatory capture, protecting incumbents rather than slowing progress. The open-weights mandate, he says, would cut funding for future training runs without any regulator deciding anything, and would force Anthropic, OpenAI, and others to publish their models' weights on release day.
Any AI model a company offers to the public has to be released as open weights.
- Aurornis
This is incoherent. The argument seems to be that releasing the weights would slow the frontier labs from raising money, which would give them less money, which would slow progress on AI. It assumed all labs in the entire world agree to self-destruct this way and no new labs every start up to continue the work.
No, really:
> As you know, funding for frontier model development depends on valuations that assume the weights remain proprietary. By changing that assumption, we can reduce the money available for future training runs and slow progress at every lab at once, without a regulator deciding anything.
Which doesn’t even follow. It depends on every lab in the world agreeing to self-destruct their valuations and stop competing. That’s a much more impossible ask than anything Dario is asking for.
It also ignores the possibility of new labs starting up, using the open weights, and continuing exactly where the old labs left off.
I feel like I just read the ramblings of someone who got so excited about the headline that they didn’t take the time to think if their argument was sound.
- tristanj
The author put zero rational thought into this. If Anthropic is legally required to release the weights of any publicly accessible model, their logical response will just be to stop offering public models. Anthropic gets most of its revenue from enterprise agreements anyway. Providing public models or providing public access to their models is their secondary business. They can cut it off any time. Public access to their models (at subsidized rates) has caused so much indirect harm and financial cost to them that they're very well justified to stop providing access.
Anthropic has already shown they're very comfortable with holding models from the public. Mythos has been around for what, six months, and there's never been a public release. Only the select few blessed by Anthropic have access.
Forcing transparency on public models will just accelerate privatizing access to these models. You don't want to live in that world.
- jsnell
> Any AI model a company offers to the public has to be released as open weights.
The obvious outcome of this plan would be frontier models not being released at all to public It'd be an amazingly bad outcome. Models accessible to the public would stagnate (no capital, no access to frontier model tokens to distill from), while internal models would keep improving at their previous pace, use them in-house, and eventually eat the whole economy.
- throwaw12
I am very skeptical of those 3 faces of American AI labs: Dario, Sam and Elon
Are they trying to stop everyone else from catching up with them or have they hit some kind of roadblock to improve models even further?
But IMO, they're not trying to help humanity
- bryan0
Dario has been pretty clear and consistent that:
1. AI model safety is the most important thing.
2. open weights decreases model safety
So it seems unlikely that this suggestion would be well-received