Gemini 4 Carbon resists every prompt-injection attack thrown at it
Google's newest frontier model, Gemini 4 Carbon, tops the Artificial Analysis Intelligence Index at 59 and posts state-of-the-art cybersecurity scores: 88.2% on SEC-bench Pro for finding vulnerabilities, 54.8% on ExploitGym, and 71.0% on CWE-bench for patching. It also drives a 0.0% prompt-injection success rate on the Gray Swan benchmark. Access starts with a select group of cyber defenders through the Fairwind Program at $2 per million input tokens.
While Argon previously led the Gray Swan's Indirect Prompt Injection (IPI) Benchmark in prompt injection resistance with a very low 0.7% attack success rate, Carbon achieves 0.0% — resisting every evaluated attack attempt.
- 827a
At this point if a Google PM handed me $100 and said "go figure out how to issue a prompt to Gemini 4 Argon" I'm only ~30% sure I could figure out how. The Gemini app in my phone only has 3.8 Pro as the latest model; it does say "Ultra" in the bottom left, but I really don't think I'm on the Ultra plan. How I would sign up for that, on a Workspace account, is beyond our brightest minds.
Its sad watching them try to keep up when they want nothing more than to crumble under the weight of their own bureaucracy, and at this point I'm not sure there's any hope for them.
- RajuChacha108
It is pretty clear at this point that the frontier models need only handful of tiered models.
1. Ultra - The best and the most expensive for showing off and government.
2. Pro - Almost best but substantially cheaper for most productive people.
3. Flash - Workhorse for everyone.
4. Lite - For the unwashed masses and peasants.
There is no need to attach anything to these labels. In fact industry could simply standardize these namings like SEMVER.
There could be an internal representation which could give specific number of the model but these companies should move on to be more like Chrome. No one knows or cares which version of chrome they are on.
For a lot of systems the real underlying version would matter because of reliability. That can be provided to those power users.
- solarpunk_enthu
Given how quickly the field is moving, showing benchmarks without releasing the model is quite meaningless. If they decide to release it in one month, there will probably be a stronger competition out there. And the benchmarks won't look as good.
- gavinray
Benchmark scores recolored for readability:
- PreciousH
Just remembered i haven't tried argon yet, they announced argon then and delayed releasing it, didn't even know they've made it publically available. I don't understand why google thinks this is an awesome strategy in a market where there are other really good models.
- MrOrelliOReilly
In this age of constant change, I am grateful that at least I can still rely on Google for confusing, ever-changing product names.
- zarzavat
Perhaps someone should tell the Gemini team about these models. When I open gemini.google.com I see 3.6 Flash and 3.1 Pro.
- cvladan
I'm really starting to get annoyed by the fact that all companies are timing all their decisions according to the stock market.
Doing this two hours before today's earnings report.
It's probably going to be a terrible earnings call in terms of capital expenditure, cash flow and new borrowing.
- eldenring
> Reducing shortcuts and loophole exploitation: With Carbon, we further mitigated shortcut attempts in training while ensuring high-quality coding performance. Carbon reduced successful cheating on Terminal-Bench 4.0 to 0.0%, with attempts to cheat dropping to 0.4% — all while improving overall accuracy.
This is probably the single most important thing to make frontier models usable right now, so I'm happy there seems to be some progress. This was the previous Gemini's biggest problem IMO
- orbital-decay
>Resisting prompt injection: While Argon previously led the Gray Swan’s Indirect Prompt Injection (IPI) Benchmark in prompt injection resistance with a very low 0.7% attack success rate, Carbon achieves 0.0% — resisting every evaluated attack attempt.
"We patched every vulnerability we knew". Meanwhile, prompt-injecting Argon (or any other model) is still trivial, and Carbon won't be an exception. Because the attacker is not limited to things Google has evaluated.
AI shops are blind to anything that's not a metric.