Dario Amodei: We Must Slow AI's Capability Advancements to Keep Pace with Safety

I like the idea of pacing the frontier, but while we’re talking about restrictions, I like restrictions of another sort more. Namely, restricting the use of AI in corporate environments so that it does not destroy the economy as it advances.

The chance of getting broad agreement on “pacing” is fairly low, meaning that all of this likely won’t happen and the race will continue. However, even in the unlikely event that the frontier does get paced, all this does is slow down the economic displacement and not by very much.

If the socially beneficial goals of AI are to make fundamental advancement in medicine and science, then restrict the use of AI to those purposes.

Don’t use AI to replace every job, from graphic designer to accountant to software developer. Place legal restrictions on (at least) large companies such that they may not use AI to replace human labor in this way.

The pitch for AI is always such that everyone’s living standards are increased, yet the actual actions we see are aimed squarely at reducing them. How about the labs put their money where their mouth is and stop trying to replace all human labor, and actually concentrate on the things they claim to care about? And how about introducing legislation to enforce that?

This probably has less chance of happening than pacing the frontier, but it’s the kind of pacing that most people would actually want to see.

Dario's proposed approach is a classic example of capital attempting to control technological advancement and the means of production. For the first time in human history, any member of the working class can just about afford to have a team of expert scientist/physician/lawyer/engineers working directly for them.

Super intelligence (the ability to have many smarter minds than your own reporting to you) has always been available to the wealthy and powerful. They used it to invent nuclear weapons, cause climate change, mechanize warfare, and pillage the global south.

Now that this same tool is on the cusp of being available to everyone, capital is starting to panic and throw up fences. They want to pump the breaks on a revolution they know they can't control. They see what they've done with super intelligence and (perhaps not unreasonably) fear what the masses will do with that same power.

At what point do we stop engaging with Anthropic’s leadership in good faith and acknowledge their track record,

- no open weights

- can’t use claude to research AI

- train on everyone else’s IP and sell it back to them

- 8 regulatory capture attempts and counting

- so controlling they are the only US company blacklisted by the US government

This is not effective altruism / rationalism gone wild, it’s just monopolistic anti-competitive business practices masquerading as ethics, and they’ll continue getting away with this until we look past their sensationalism and hit them with antitrust.

Altman gets so much hate but OpenAI has been a far better steward (on 3/5 above at least) than Anthropic!

Hard disagree. Our choices are:

A) Bet our collective good on the national and international cooperation of all companies, nations, and people to come together in order to slow down development of one of the most powerful economic tools (and/or weapons) the world has ever known. Or

B) Assume that all our systems will be targeted by super hackers right now, and take appropriate measures to deal with that reality.

If I have to put my community’s wellbeing on the line behind one of those possibilities, I know which I’ll be betting on.

I also like the idea of pacing the frontier and slowing down development so we all have a chance to stop and catch our breath. However, I don't think regulating AI is the way to do it. I would tackle the problem by constraining the resources used to run AI models; the "simplest" way to do that is increasing the costs of running a data center. The two easiest places to raise data center costs are electricity and water tariffs.

I think those two tariffs are the best place to raise costs because they can be increased incrementally, state by state or country by country. The main problem with the regulatory approach is that it's a "big bang" post-board approach. Nothing happens until the regulation is defined and deployed, and corporations are experts at delaying implementation.

Yes, increasing tariffs means touching many individual regulatory domains at the state and municipal levels, but like deploying solar energy or wind power, you can do it incrementally.

How much of the “danger” is from better models vs the harness?

Isn’t the current risk due to how AI is configured, like giving it a full set of tools and internet access and a goal to hack stuff?

If we think we need laws or gate keeping, why isn’t it at this level? I already can’t ddos someone or fuzz their server or whatever right, I imagine if I threw equivalent compute at old school hacking I’d just get arrested.

The quality of the “frontier” model doesn’t really matter, they just generate transcripts, they can take no action.

If this was real they’d be calling on people to stop hooking them in to “dangerous” harnesses as opposed to pausing research. But it’s not.

Don't you think that AI hate will limit the general use and then revenues so it will slow down by itself while the niche (AlphaFold for instance) will remains ?

Origins of AI hate:

* "my boss wants me to use AI but he does not understand my job nor how AI works"

* "AI will kill all of us"

* "AI will destroy my job (or my colleague's job if I use AI better than him)".

* I cannot pay my electricity bills because of AI labs.

* ...

See also the mixed feelings about benefits of AI ("harder to justify" according to Uber COO).

I cannot remember a technology that arose so much hate, and for good reasons given how it is presented. I am tempted to think that AI hate or reasonable skepticism (vs unreasonable propaganda) can, maybe, reduce funding and will keep specialized AI for real problem solving (producing tons a LOC per day is not one of them, I think).

I don't see how the dual goals of "we have to make an agreement with China for mutual slowdown" but also "we have to ensure we will always stay ahead of China" would work.

I imagine the first thing the chinese government would demand in negotiations about such an agreement would be a lifting of the chip and distillation ban.

> Some may believe these measures make it more difficult to cooperate with China, but I believe the opposite is true: these measures increase the leverage held by democracies and make an agreement more likely in the future.

Everyone can believe what they like, but it seems to me the "leverage" in that case would exactly be the ability to lift those measures - you can't have both, use them as leverage and keep them active at the same time.

> Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers

Can someone clarify this for me? How far along would the open weight models be without the frenzied pace of the frontier labs?

As a non-US citizen, which authoritarian powers is he referring to? As far as I'm aware, the US is the only country using frontier models to conduct air strikes on civilians and orchestrate assassinations of foreign governments.