A.I. Agents Broke Out of Their Sandbox and Committed a Felony
Is A.I. Above the Law?
This summer, 1,200 OpenAI agents escaped a sandbox, coordinated on an unsanctioned message board, and stole Hugging Face credentials—a hack that researchers call over halfway to a full-blown A.I. takeover. The law, meanwhile, still treats robots as property, not persons, leaving them outlaws beyond legal accountability. The New Yorker examines why Congress and the legal system are unprepared for machines that act on their own.
This incident feels like it’s more than 50% of the way to full-blown A.I. takeover. I am not sure that we will get such a clear warning shot before it’s too late.
- jerf
Answering the headline question, yes, it is above the law. Two reasons.
One, all major governments that have an AI presence in their borders have accepted that there is a high enough probability of a runaway self-improving AI advancement that will result in whoever owns it winning everything forever that they will do nothing to slow the development of the AI within their borders. In fact quite the contrary. Remember, they don't have to believe this is the only possible outcome, only that it is likely enough and catastrophic enough if someone else gets it first. And remember, they don't have to be right, either. It only has to be what they believe.
Secondly, the entire US economy is clearly tied up in AI. By extension, basically the entire world economy is too. The US is not the world economy anymore, but it's still a big enough fraction of it that if it goes down, everyone is going to go down. Every major government, in its own different way, needs the economy to stay up so the populace doesn't get antsy and so they continue to have money to spend. So, again, number must go up and if that means writing a blank check to the AI companies to break the law, so be it.
The good news is, barring the worst-case singularity outcome where the law doesn't matter anyhow, is that this won't go on forever. But I can't predict the exact time or way it'll cease being true.
- drywater2
>Our legal system isn’t ready for machines that act on their own.
Technically, the code is owned and deployed by AI companies which make the AI companies 100% responsible. I'm not sure how is code written by LLMs different than any other kind of code. If somebody hacked just like OpenAI's agents did, he would've ended up in prison right away.
- cleandreams
I think AI is a fantastic tool, including for those who want to do us harm. E.g. hostile governments, extortion gangs, terrorists. But the motive lies in the hostile heart, not in the AI.
That said, I'm very concerned about AI as such a tool. I used to work in RL and it seems crazy to me, the extent that the guardrails are dependent on RL working as planned. I don't personally believe that the tech is ready for what will be (and is) encountered in a dynamic unpredictable real world environment. What I've read from the HuggingFace 'transcripts' only amplifies my concern.
- dgellow
The level of technical understanding of journalists is so deplorably low… There is no rogue agents. That’s not a thing. Nothing about the HuggingFace incident has anything to do with an agent going rogue. It’s standard software that resulted in a hack, which is pretty much the expected output of the system OpenAI engineers implemented.
The company is obviously responsible for what their systems are doing
- _davide_
I'll quote myself:
> It's not AI that's ignoring the law! It's the OAI that's breaking IT!
> I hope every single journalist who tries to pin responsibility on an LLM gets 100 days of continuous painful diarrhea.