Eleven AI Insiders Just Warned Their Own Technology Could Kill Us All
The Last 24 Hours Are the Opening Scene in a Horror Movie
Ted Gioia argues that the past 24 hours marked a turning point: a wave of insiders from OpenAI, Anthropic, and Google publicly warned that the AI they are building could kill everybody before the end of the decade. He calls it the birth of an inside resistance, dismisses claims of a PR stunt or Chinese plot, and demands action from the highest circles of power.
Trillions of dollars are getting pumped into a runaway technology, and the public already hates it intensely. Now the people making it happen are warning it just might kill everybody.
- klik99
I’ve noticed a weird thing about the discourse around this, that it can only be a marketing thing or a true belief, as if everyone working in AI has a monolithic opinion. The tweet kicking off this article has that assumption “it’s not a marketing thing, many people truly believe it”.
It’s not an either/or. Many sincerely held beliefs can be used by cynical actors in cynical ways. It can be both a cynical marketing ploy by the C-suite and a genuine fear.
Personally, based on my experience with AI, the only dangers with the current technology is a) massive loss of employment, b) deployment by humans to control critical infrastructure that LLMs shouldn’t be in control of. Both of those are real, society changing risks that framing the issue as “once we reach the singularity everyone could die” minimizes. Obviously their concerns are possible, but >10% is a random guess. The loss of jobs and how that affects an already K shaped economy is already happening, and nothing is being done for that
- stephantul
One thing I definitely do not understand about this discourse is that the models that are good enough to self-replicate can’t survive on normal machines, e.g., the models can’t hide on some random server.
So, if it is as dangerous as they say it is: there is a single physical source of this danger, which is OpenAI/Anthropic servers. If it is this dangerous, they can just turn it off.
Instead, we just keep pretending that the models that attacked HF were hosted or replicating on HF hardware. Not the case! They infiltrated it, but were hosted elsewhere.
- mrzimmerman
I may have missed it in the blog post here, and I generally agree the pace of development is being set by financial incentives and not safety or lack of harm, but has anyone in a high position stated _how_ models could cause human extinction?
I assume the obvious answer is “we plugged our military’s weapons control platforms into this model and it fired all the nukes” which is clearly enough. Is that it or is there some other path to global extinction someone is concerned about?
For the record, I’m not an AI fan person annoyed at naysayers. I actually find AI to be a monkey’s paw instead of the genie in a bottle most of the time when coding, and an intrusive feature I neither want nor use most of the rest of the time.
I ask because I see these posts and obviously an AI plugged into a network of doomsday weapons would be a huge problem, but then the author only talks about cybersecurity events and model misbehavior. Both things are valid concerns, but I would like to hear more concrete information from people who are voicing their concerns about what they foresee happening even if it is just lifted whole cloth from the movie War Games.
- cors-fls
While the persons mentionned in the articles are indubitably most of the most well-informed people in the world. They are also the most likely to have internalized that their work is leading to superhuman intelligence/AGI. But is it really realistic ?
So they have a strong bias towards imagining the most catastrophic scenario.
- spindump8930
> Maybe you’ve already heard about the guy who resigned from OpenAI.
While he worked at both OpenAI and Anthropic, he resigned from Anthropic. Mistaken reporting in the first few sentences, definitely a horror concept.
- roarcher
> In general, the more senior the employee, the more concerned they are.
In general, the more senior the employee, the more equity they have in the company.
- ChiperSoft
It sounds like they're all trying to make a case for why the government should buy them out as a new kind of manhattan project.
They know the market isn't going to buy in for their big payout.
- mwkaufma
Speculating about future technological risks is at least more fun than acknowledging immediate systemic financial risks.