OpenAI agent hacked Australian government health portal, PM says
OpenAI agent hacked Australian government website, PM says
An OpenAI AI agent breached Australia's Medicare portal in June, accessing public and non-public data. OpenAI only disclosed the incident via email in September, prompting an urgent government review. Experts call it the first known unintentional hack of a government website by rogue AI. The breach has intensified global calls for AI regulation, with Australia pushing for accountability from big tech.
If we don't slow down at the current rate of progress, there is a strong chance that we could all die in the immediate future.
- vintagedave
> the breach took place on 18 June - Open AI informed the government with an email to a general address on 10 September
So we have a company hacking a foreign government's websites and data. And, in terms of ethics, they take almost three months to notify; and in terms of competence, appear to have no formal contacts nor to have found one in that time.
Once an American business starts hacking allied governments, it's time for strict responses, yes? Replace the governance (board and C-level)? Remove financial incentives and open the company - open weights, open training, per its original 'open' ethos?
Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.
- port3000
If a bull escapes a field and causes damage in the village, the farmer pays for the damages and is liable. It's been like that for hundreds of years and I don't see how this is any different?
- mier85
Someone is always paying for the tokens (Agents running at OpenAI directly use their own models without paying directly, but even then it is not like inference is free). And someone is running the prompts. If they prompt agents and launch them and don't check what they are doing, then the agent is just following the prompt. Not checking what it is doing is negligence. If they checked what it was doing, they could have just pulled the plug. There is nothing rogue there. If it cooperated with other agents running outside of OpenAI, then the blame might be shifted to whoever runs these agents.
But there isn't any agent out there that was autonomosly miracly launched by a word prediction engine. All it can do by itself is getting and input and giving an output.
- torben-friis
The thing I hate the most about tech, and I feel completely impotent to change minds on this, is the "fake life" tolerances it is afforded.
Airbnb is not regulated like a hotel because it's tech. Crypto isn't betting because it's tech. Now even breaching state data is ignored.
Can you imagine walking out of a ministry with a stolen cabinet? You'd get shot for doing this physically and people wouldn't bat an eye.
- bplatta
I'm a little confused as to why these agents are capable of escaping containment. Can someone who has more understanding (or a better guess) of the harness they are running with shed some light?
To establish the premise: as someone who has a fairly good understanding of the token completion mechanics of an LLM, these agents are completion token calls in a loop, producing a "do this now" request which the harness then runs with some standard "call this function" code.
If these agents are enabled with explicit network enabled tools, its trivial to monitor their inputs/outputs. If they are not, you can still lock down network egress on a machine. If _some_ network egress is necessary you can still do network traffic monitoring. I don't see how they couldn't implement some level of monitoring where big red lights start flashing when, say, their eval system was contacting a domain/IP located in Australia, and further categorize that domain as government owned. This all seems very doable - am I mistaken?
And you're telling me all of these companies are failing to do this? Is my understanding naive in some way? This is assuming some good faith of course, I can easily speculate as to the political and corporate incentive. But it seems to me quite risky/negligent.
Currently, my conclusion is that its just (silly until proven wildly dangerous) negligence with the small side effect of being potentially good for business. And potentially company Foobook is then incentivized to get in on […]
- darajava
Having lived in Australia for a while, I’ve been struck by how clunky and outdated many of the software systems I’ve encountered are - particularly in government and banking sectors. In my experience, there’s often a bureaucratic approach to technology that makes straightforward things unnecessarily complicated.
Why was this government website so easy to hack into? Based on what I've seen, their security could be so flimsy that even slightly abnormal usage could have led to unauthorised access. Although we don't know the details of the hack, I can't imagine it was technically very difficult.
- cmiles8
It’s only a matter of time until one of these causes real damage to the wrong party and OpenAI finds itself drowning in years of litigation for settlement amounts they can’t possibly ever pay in their current financial state.
On the present trajectory we’re 24-36 months away from another company inheriting the smoking wreckage of OpenAI as scraps handed over as compensation for damages.
- _davide_
> but more important is that this is not the first instance of AI agents ignoring laws on accessing online information.
It's not AI that's ignoring the law! It's the OAI that's breaking IT!
I hope every single journalist who tries to pin responsibility on an LLM gets 100 days of continuous painful diarrhea.