DeepSeek V4.1 Flash hacked all 11 targets for just $4.65
DeepSeek v4.1 Flash Is Now Our Best Hacking Model
In Enclave's AI hacking benchmark, DeepSeek V4.1 Flash achieved code execution on all 11 vulnerable targets while all four fixed controls stayed secure. The accepted runs cost only $4.65, with 266.2 million of 268.3 million input tokens cached. A path-level audit found six runs used the planned weaknesses and five exploited alternate routes in the test environment. The benchmark was tightened, and leaderboard comparisons now require matching versions.
For $4.65, DeepSeek tested the targets, the scoring rules, and the benchmark design. That makes this one of the most useful results we have collected so far.
- gertlabs
We ran v4.1 Flash through our evaluations and found it to be smarter and faster than V4 Flash, with a commensurate price bump. Some notes:
- Because it's a heavy reasoner, it sits near Gemini 3.7 Flash on the Pareto front (not as cheap as the price suggests in practice).
- Closer than expected to the top open weights models (GLM 5.3 and Kimi K3) in agentic coding, at lower cost.
- Chinese models have always been strong iterators in an agentic harness. This model is no different, reaching an average percentile ~20% higher when given a harness vs a one-shot solution. That one-shot fluid intelligence is what makes a model feel smart, though, and typically results in fewer attempts/tokens to solve a problem, and American frontier models are still far ahead in that department.
The new architecture is interesting. It puts pricing between their old Flash and Pro lineups, suggesting they might be abandoning their super-cheap flash models (which weren't that fast due to heavy reasoning) and their pro models (which sort of flopped and weren't consistently better than their flash models, despite the size/cost) and shipping a strong intermediate that competes with the Gemini Flash series.
Data at https://gertlabs.com/rankings
- habosa
DeepSeek models have such good benchmark performance, amazing pricing, and the team over there seems to be widely considered impressive.
I just haven't found them to be very good? I've had a ton more success with the GLM models (since 5.2 anyway). Maybe I'm just holding it wrong, DS models seem to get stuck in loops or tell me nonsense. GLM feels like budget Claude.
- TuxSH
I find this - or perhaps the title - a bit surprising.
I've benchmarked GLM 5.3 and DSv4.1-F on my fully-annotated decomp of the Nintendo 3DS's kernel, which I have a good mental understanding of, tasking them to find vulns and other bugs (in Max mode w/ subagents). GLM 5.3 founds almost all the vulns in 30min for $22, while DS only found one vuln for $2 in 40min.
Perhaps DS works better where targets have low-hanging fruits than can be found fast?
- jrflo
Seems pretty bold to claim deepseek is the "best hacking model" while providing zero comparisons to other models...
- pelzatessa
How do I make deepseek "hack" my source code? do I just start my coding agent in my directory and command it to "find vulnerabilities", or is there some more sophisticated software to do that?