Anthropic researcher resigns, forfeits equity to warn AI could kill us all
Good people refuse to do bad things
Jacob Coxon, a pretraining researcher at Anthropic, resigned two months before his equity vested and went public with a stark warning: the people building AI earnestly believe it could kill everyone by the end of the decade. His posts drew over 115 million views and a response from an Anthropic alignment lead who put the risk above 10 percent. The article compares his exit to Frances Haugen's Facebook whistleblowing and argues that good people refuse to do bad things.
The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt.
- cogman10
I disagree with this article and here's why.
Life isn't black and white with "good/bad". Further, there's no objective measure of what is good or bad. Everything is a shade of gray.
You certainly could argue that some actions are absolutely good or absolutely bad, but those aren't the majority of actions people take on a day to day basis. People aren't often faced with "should I kill this person" or "should I publish the cure for cancer". Instead it's "should I eat a hamburger or a salad" or "Should I finish this jira case or that jira case".
For example, you could argue that anyone working in an automotive plant is doing something bad because they are helping to produce vehicles which burn fuel which perpetuates climate change. But those plants also often produce parts for farm equipment or vehicles used to transport necessities like food.
People working at AI companies can make similar personal calculations. Like yeah there are bad things happening because of AI, but also they can believe that there are good things that can come as a result such as research and product developments.
- oconnor663
> do not consider yourself a contributor to humankind
Good lord. You could apply the same logic to any taxpaying resident of any country whose government is doing something bad. There's some truth to it, it's worth bearing in mind, but it's also the very definition of an Isolated Demand for Rigor.
- ArcHound
Generally, I agree.
Imagine: you start working at the company with a good will towards them. As you learn more and see more decisions, you slowly realize how bad it is. Now you've already contributed. Then you leave.
It might be possible to build everything with this goodwill and that's how we'd still end up with a torment nexus even if you immediately drop it.
What I'm saying is, at one point you have to break the company fully.
I'm not sure what more we need to wait for, if the claim "10% it ends humanity" is not enough.
- tptacek
The author of this blog is an AI skeptic (see the backlog of posts). That's fine, but it's hard to square with this particular post. To take Coxon's argument seriously, you have to believe frontier models are hypercapable, and not just hype objects. That's what Coxon believes and why he left.
- magimas
>The accusation
>So, why are you still working for them?
>If the answer is the salary, say it louder, and do not force yourself to make excuses.
>If the answer is “someone worse would replace me”, say it too, and louder too.
>If the answer is “I can change things from the inside”, then show me what you changed. Haugen tried the inside. Coxon tried the inside. At the end, unfortunately, the inside did not move.
>
Okay, but what if I think that both sides are LARPing their personal science fiction stories and reality will end up much more mundane (but still incredibly beneficial) than the utopias and dystopias everyone is cooking up?
This is presupposing a moral dilemma that just doesn't exist in my opinion.
- Aurornis
> More than half of tech workers (53%) would steer a newcomer away from their own field, and they stay.
It would be nice to see this survey repeated across professions. In my experience, most people have gripes about their profession because it’s what they know. Only half of respondents saying they would steer newcomers away feels a lot better than what my friends in medicine say all the time. Among my college professor and professor-track friends, this number probably reaches 100%.
- bravoetch
If everyone doing a 'bad thing' stepped off the hamster wheel today, the world economy would collapse before markets closed. The biggest companies have built an attention and surveillance nightmare machine that funnels all the money to a relatively small number of shareholders.
- fi-le
It's refreshing to see a well-argued and balanced essay on AI risk being upvoted on Hacker News.