Google Fixed More Chrome Bugs in June Than in the Past Two Years Thanks to AI
Google fixed more Chrome bugs in June than over the past two years, thanks to AI
We are leveraging Large Language Models to revolutionize how Chrome discovers, triages, and patches security vulnerabilities. By deploying AI agents at scale, we identified a sandbox escape that survived for over 13 years and fixed more bugs in recent months than in the previous two years combined. This automated approach accelerates our response to threats while maintaining rigorous safety guardrails to protect user data.
One of the bugs we found was a sandbox escape that would allow a compromised renderer to trick the browser into reading local files—a bug that quietly survived in our codebase for more than 13 years!
- VBprogrammer
I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change.
In fact, if it wasn't for the fact that it made making the actual changes I identified much easier (move these joins into a CTE etc) it would have been a detriment. Not only did I get sidetracked by a bunch of useless suggestions but I also had to put up with others dumping their raw AI output at me as if it was somehow a meaningful contribution.
- unprovable
The real datapoint was Firefox not paying any money in Berlin's Pwn2Own competition round this May just gone. Unheard of to have nothing confirmed... they've paid out every event since 2007 (I checked). Does this mean we must move past the low-hanging fruit now? Probably... Certainly indicates some usefulness of these models.
- truncate
Not that I don't believe its possible to fix a lot of bugs, I also wonder what the actual dynamic was. Were the people in team working much more than usual as well? Given its Google, I wouldn't be surprised if there was an "internal push" to fix more bugs over next X sprints so that they can publish this blog and some manager can show impact and AI adaption to his superior.
- glimshe
A lot of people here seem to be living in a different universe than me or simply don't know how to work with AI. I think detractors believe you should just let AI do the job blindly instead of leveraging it as a tool to accelerate you. They get mad at Excel for the poor investment returns. At this point, this is such a strawman, it isn't worth counter arguing.
I think I'll abandon this discussion and keep using AI quietly while exchanging tips with like-minded people who are interested in using it properly and efficiently.
- dabedee
How many of those automated fixes were reverted? How many introduced a new bug? What's the false positive rate on the finding agents? The post has counts for everything that went right and nothing for what could go wrong.
- Phemist
Worrying. Extrapolating a (speculative) future, this means (Google will feel) that soon the chromium base will no longer need the crowd-sourced bug hunting that is open-source. I expect Google to eventually stop working on chromium (in the open) and all current chromium-flavours will become de-facto forks of the last published version of chromium. These forks won't be equally easy to maintain given that the groups running them do not have access to the same level of subsidy as Chrome does with Gemini.
- mw888
AI critique often funnels itself into a narrow bucket: creating code blindly with AI is bad. That's easy to grant.
Adversarial testing, checking developer assumptions, refactor suggestions, small dev tools and even some guided coding all sit on the other side of the spectrum of what you can do with coding and AI. For larger and larger codebases even simple things like tracing dependencies or behavior might be greatly aided.
And the critiques reserved for that narrow bucket on the other end, blindly generating code, are too easily conflated with the rest.
- goldenarm
Elephant in the room : how many of these bugs were written by LLMs in the first place ?
Because creating 100x more bugs and fixing 100x more isn't something to be proud of.
- andai
This situation seemed familiar... https://i.ibb.co/RGvQgfmX/i-fixed-more-bugs.png
- MSkill1
Did they fix the bug where Chrome tries to track you and your behavior wherever you go and whatever you do in the world?