A 2020 Breach Exposed 200 Million Brazilians. Now AI Agents Are Training to Find the Same Flaw.
The systems that no one will test
In 2020, I found a vulnerability that gave access to the Brazilian federal system, exposing records of over 200 million people. I reported it and it was fixed quickly. Now, with AI agents being trained in RL environments, I worry about the government systems that will never be AI-pentested. The flaw I found required only attention to detail, and that's exactly what worries me.
I also can’t help imagining that, somewhere, there is a very similar environment that reproduces what I found in 2020: almost the same system, almost the same flaw. Multiple agents enter it thousands of times a second, generating trajectories until one of them finds the flaw. The difference is that when this is put into use, in none of the attempts does anyone pick up the phone.
- Modified3019
Relevant:
https://www.reddit.com/r/YouShouldKnow/comments/1wssf4u/ysk_...
https://imgur.com/a/6FaQQhb (original post before being taken down by mods)
>The Work Number is an Equifax (who famously had a massive data breach a few years back) owned database with employment and pay information.
>You can create a login on theworknumber .com and pull your report. Mine is seventy four pages and had info from every company I've worked at in the last 13 years including every paycheck I had received with the exact dollar amount (both net and gross) and hours worked. It has employment start/end dates, termination reason, details about withholdings, benefit enrollment, union affiliation, etc etc etc.
>This data is sourced directly from the HR platform your employees use (ADP, Rippling, Gusto, iSolved, etc). Equifax sells your data for things like employment background checks.
>You cannot have your data removed from The Work Number. You can freeze your report (much like a credit report) but if a potential employer cannot access your frozen report that may disqualify you.
>Why YSK: If you're interviewing for a job you can be at a significant disadvantage especially for things like salary negotiations because they can literally see how much you make including your most recent paycheck. Creditors also can access these reports.
>This is the biggest privacy violation I have ever seen and almost nobody is even aware of it. Certainly none of us consented to having our employment and income data h […]
- areoform
The narrative around security and LLMs doesn't make sense to me.
I think we've created a self-fulfilling prophecy. Everyone involved is acting with the best of intentions, but in avoiding what they fear, they've give shape and realized their fears. Much like a greek tragedy.
An example of this is the story of Oedipus Rex, in the story Laius, the king, is told that he is "doomed to perish by the hand of his own son." (and wed his mother) And so to avoid this fate he decides to kill the infant. The person assigned to abandon him in the woods takes pity on the baby and gives the baby away. Thereby ensuring that Oedipus knows neither his mother or his father (and arguably giving him a reason to kill his father).
The child grows up and hears the same prophecy again and the child tries to avoid the prophecy as well, as he loves his adoptive parents. So he leaves them and travels to Laius' kingdom, where he runs into Laius. Neither recognizes the other. As Laius is the type of man to kill an infant, they end up in an argument, whereupon Oedipus kills him.
I think the ancients were on to something, because if Laius had reacted to the prophecy with courage, he would have been saved. I would like to argue that if he had faced his fear and raised Oedipus with love, then the necessary preconditions for the prophecy to come true wouldn't have taken root. But that's not what happens.
By being driven by his neuroses and in acting with cruelty out of fear, Laius makes the prophecy real.
To […]
- teiferer
> I honestly don’t blame them: [...] software is software
That attitude is the problem. Why does our industry have that attitude towards quality? Every bike shop in my little town is better with quality than the average software shop in the world. Yes, software is more complex than bicycle. But a software engineer also gets paid 10x and has the luxury of spending substantial time on their product, compared to the 10 minutes it takes the bike guy down the street to diagnose and then fix an issue with my bike which I then trust my life with once they are done and I bike through traffic.
We need to treat software differently. "Oh well, it's just software shrug" does not cut it anymore, if it even ever did.
- kitd
Can't escape the feeling that the original vulnerability was deliberately planted as a mechanism for law enforcement (or maybe criminal orgs) to "expedite" investigations.
- madaxe_again
Is it just me that has a constant dribble of other people's private data coming to me from government agencies?
This year, I've had the UK variously send me someone else's court summons, someone else's tax documentation (not address error - just inadvertently included as attatchments on emails to me).
I've had portugal disclose property taxation information on any subject to me, through a simple enumeration hole.
And last week I had brazil's ministry of agriculture send me details on other peoples' livestock shipments.
Our information environment is unbelievably porous. There would be no secrets from an unshackled AI.