Dario Amodei: AI regulation isn't a choice between concentration and chaos

> We are however doing our best to fix this: Anthropic is ramping up its efforts very quickly in biology and medicine, and we hope to have incredible results in the coming years and some early glimmers in the coming months. When we’ve actually accomplished something real, the whole world will hear about it, as loudly as possible, you have my word on that.

1. He is giving his word that if he cures cancer, he will brag about it as loudly as possible. Admirable dude! I actually trust him on this 100%

2. “We hope to have incredible results” -> doesn’t everybody hope to have incredible results?

3. “Some early glimmers in the coming months” -> is that planned for a few days before the IPO?

This guy says less than politicians do.

I genuinely think Dario is a well intentioned, intelligent dude, but I think him and Anthropic have a huge PR problem and are really out of touch with how they're perceived.

Anthropic in particular has developed this almost Orwellian like veil of condescending rhetoric that on the surface suggests they're looking out for you while underneath they're taking actions that suggest they do not trust you, Mr/Mrs Ordinary Person. All the safety rhetoric, never supporting open weight models, the lockdowns on harnesses outside claude code, etc.

Anthropic if you care about public good, do something to empower people. Release an OSS model. Open source Claude Code. Open source some inference tooling or something. Just give people anything except your words.

> Overall my view is that AI is structurally a technology that tends to concentrate power, for reasons that have nothing to do with regulation (more to do with the extreme implications of the scaling laws). Open-weights do help some with this but are nowhere near a sufficient solution because they simply shift the concentration somewhat to those with the most compute and chips (which are roughly the frontier labs plus maybe hardware providers).

This is actually a good point. Open models are getting better and better, some of them might even be useful in consumer hardware now. But if AI performance is still correlated with compute power, then no doubt power will remain with the people owning the chips.

Just want to say, this comment section on this website would have been unthinkable a few years ago. It’s wild: all the engineers have developed class consciousness. I wonder how that happened?

Anyway, it’s fun to watch, but remember to stay sane guys, be aware of the pendulum dynamics.

I am skeptical of David. But he is spot on with this comment:

“Yet Dario blames public negativity on a long-standing loss of trust in institutions rather than his own messaging.“

Dario’s arrogance and blind spots are really something to behold.

I don’t think my people got my previous post but apparently a lot of diseases were on track to being cured regardless of the invention of LLM based AI. One thing that bothers me about this technology is it makes it easy to attribute all effort to it. This not only hurts the ego but also makes it difficult to reason about how effective it is compared to how effective the people using it are.

Secondly, if you look at what Dario has actually written, then the cure to the big conditions like cancer is in his view hopefully going to be a result of scientists working 10x faster and therefore making big Nobel prize paradigm shifting discoveries 10x faster. So Anthropic suggesting “we are going to cure cancer” doesn’t make a lot of sense to me. According to his original vision it looks like a bunch of groups now using AI independently producing breakthroughs which cure cancer.

One thing Dario said is important reflecting on (but from another angle): where's the big deliverable from AI? If AI makes us 10x more productive, the 3 years since its popularization were enough for a product that would have taken 30 years to build without AI, for instance.

I'm an AI advocate but that question makes me feel that AI is simply "very useful" rather than being a historical game changer for humanity.

> I don’t think that a glitzy marketing campaign with a positive spin (which some have advocated that Anthropic do) is the way to win back that trust — at this point, saying that AI will cure cancer is more a cliche than it is inspiring, and most people think it is deceptive. The thing that will work is actually curing cancer.

The way to do this is with large automated wetlabs running experiments and generating novel training data on the behavior of cells.

It's with partnerships with governments and universities that educate the masses to the point where many of us are better able to contribute towards the project of growing the available knowledge in a useful way.

What's actually happening is were spewing carcinogens into places where humans will breathe them and lying to ourselves about how if we just keep going this way we'll cure cancer.

Stop making the world less habitable by trying to squeeze another drop of model utility out of the existing knowledge, and start helping with the creation of new knowledge.

It's going to take decades to ramp up. Let's turn the page and get started.

"I wrote Machines of Loving Grace because I didn’t feel the AI industry was painting an inspiring enough picture of how the technology could radically transform the world for the better. The bulk of the essay is devoted to refuting skepticism of AI’s potential in health and biology, and showing why I think it will actually be possible to cure most human disease in ~5-10 years, as crazy as it may sound to ordinary people and frankly to biologists as well (I used to be one!). And, if you read my most recent essay (Policy on the AI Exponential), I discuss concrete proposals for how to streamline the FDA process to make sure the deluge of AI-accelerated drugs isn’t slowed down by the regulatory process."

This is fantasy, and I believe the vast majority of those actually working in pharmaceutical R&D would agree. Contrast Amodei's opinion here to the Derek Lowe post I shared recently: https://news.ycombinator.com/item?id=49313367 . Derek Lowe's opinion is closest to my own experience: AI, whether it be traditional ML or LLMs, can help here and there -- the former to help you to triage paths to explore in a way that's a little better than intuition in some cases, the latter mostly to generate code faster in the code-dependent aspects of pharma research -- but neither of these things are significantly widening the main bottlenecks. I don't think the data exists to do so, especially since so much of pharma R&D is looking for higher and higher hanging fruit (i.e. exploring avenues f […]