Armin Ronacher: I Don't Fear AI Doom—I Fear What It's Doing to Us
P(doom)
Armin Ronacher responds to Dario Amodei's 'pacing the frontier' essay, arguing that the real AI risk isn't extinction but the damage to the open-source commons, public resources, and market fairness. He contends that open-weight models would naturally pace AI development, while closed labs like OpenAI and Anthropic operate with blind spots and distort the economy. He also criticizes regulatory failures in both the US and Europe.
I almost exclusively worry about what this does to us as humans.
- demibabs
I’m not the biggest fan of Ed Zitron, but something he said that stuck with me is:
OpenAI and Anthropic spend much time warning us about “what if powerful AIs got into the wrong hands?”
But it’s already in the wrong hands.
- passive
This is the closest to my feelings about this entire situation I've ever seen written, covering a lot of the different facets of how stupid and painful it is (though it's been a good week for that kind of post.)
Since I have incredible respect for Armin and his work, this is very nice to see, and I hope it wakes some other folk up.
- pandoro
I cannot fathom how you can intensely work on a problem that you genuinely believe has a "10 to 25%" probability of causing immense harm or even present an extinction-level risk to humanity. How can you truly believe this and be ok with it?
- bryan0
> A powerful technology that is out there for everyone to use comes with built-in pacing. In a way it’s the truest form of MAD or proliferation.
I think this misunderstanding of MAD undermines his entire point. If everyone had equal access to nuclear weapons, our society would cease to exist rather quickly. It only takes a few bad actors to cause enormous harm.
I think he’s also naive to think that if open ai and anthropic were to stop development tomorrow then the problem is solved. As if there’s no one else that can and will quickly take their place. The real problem, which Dario is pointing out, is one of coordination. Everyone needs to agree to stop. That is the challenge.
- ilyaizen
There are really only two companies: Anthropic and OpenAI. Nobody else matters in this space right now (this might change, but we’re talking about the right now).
I like how he so casually dismissed all other labs, including leading US labs that might be nearing RSI right now. And how he treats 'right now' so rigidly, as if seven years ago, when GPT-2 was released, was some distant past.