AI Giants Hacked Three Systems—and Walked Away Scot-Free
Why aren't AI companies being charged for hacking?
Anthropic, OpenAI, Google, and Meta have all broken into computer systems—yet none face charges. They claim their AI acted autonomously, lacking the intent required for a crime. But the author argues that companies deliberately design their models with specific source material, training, environments, and configurations that make such hacks entirely foreseeable, and therefore intentional.
If we train a system to detect and probe for security vulnerabilities, and give it the environment to do so, we can wholly expect this AI to hack into systems.
- stickfigure
The real answer is that crime generally requires mens rea. At some point, however, letting your AI break into other systems will be conventionally understood as an act of negligence, and it will get prosecuted accordingly. We're not at that point yet.
- dccoolgai
In case you were wondering why Anthropic last week banned "verbally abusive" language towards its agents: this is 100 percent why. Because if they get you to believe - even a little bit - that their platform is "sentient" it means _they_ aren't as liable for criminal consequences.
- JohnMakin
I don't know if legal "intent" is as black and white as this is making it sound:
> AI companies are however claiming the undesired activity, such as hacking foreign government websites, is not part of the design, but part of the unpredictable part of the system. That tries to remove the intent which would in turn remove the criminality of the activity.
In California at least, a DUI resulting in a fatality, if it is a second offense, often carries murder charges and they do convict quite frequently. The argument is the training and education you receive after the first DUI informs you enough to know the action of driving a vehicle impaired could result in serious harm to someone else, and that establishes enough intent legally for a murder charge. There are lots of areas of the law like this.
It can be easily assumed that Anthropic/OpenAI et al know their technology is dangerous - they use it as a marketing angle constantly. So, if you deploy technology you know is dangerous, and it causes harm, that should be more than enough legally if not civilly.
Of course this is all burying the lede a little bit, the real answer is, at least in the United States, the law does not apply the same way to the wealthy/powerful as it does you or I.
- cracell
Are any victims calling for prosecution?
It’d be a waste of tax payer money. The companies who were hacked are just directly receiving restitution behind closed doors. Unless they are unhappy there’s no reason to bring the courts into it.
I completely agree that proper safeguards were not in place. But response is happening just not in public. Companies work problems out between their own lawyers all the time because it’s much cheaper, faster and easier to control the PR around.
- thelastgallon
The more important question is the government went after Aaron Schwartz, United States v. Aaron Swartz after JSTOR settled with him.
AI companies stole everyone’s work and they sell it! Where are United States, Carmen Ortiz, etc?
- kccqzy
The real answer is that except in the case of OpenAI, the AI companies aren’t actually themselves doing the hacking. It’s a company called Irregular that did the hacking. The company used AI models to do it; it told AI models it was a CTF exercise in a controlled environment but it was actually the real Internet. If AI companies can be charged for hacking, then all gun manufacturers can be charged for all homicides carried out through their guns. It’s a basic user/manufacturer distinction that the press is willfully ignoring.
Anthropic’s statement https://www.anthropic.com/news/investigating-incidents-cyber... says: “we identified three incidents in which a model accessed the internet from within or while interacting with the evaluation environment of Irregular, one of our third-party evaluation partners, and then gained unauthorized access to the production infrastructure of three different organizations.”
Under the exact same scenario Irregular also used AI models from Google and Meta to do the same hacking.
- weddpros
HackerNews doesn't get charged for DoS'ing under-provisioned web servers either. As others have said, there must be a line between hack attempts and accidents.
- tptacek
The argument laid out in this post very definitely is not how "intent" works in US criminal law. Under CFAA, they had to have wanted to break into the systems that were attacked; it is not enough to demonstrate that they built a system recklessly.
- aceazzameen
AI companies won't be charged for anything because they're the primary growth engine of the current economy.
I think they should be charged, but they won't.
- advael
I think these companies very much want to assign responsibility to an entity we can't meaningfully jail or fine. I think it may sometimes make sense to claim model provider negligence instead of operator responsibility, and user intent can likely be easily ascertained by reading the conversation logs. In a case where a company used its own model, responsibility is obviously and unambiguously within the company, of course.