37% of AI passes on cyber benchmark involved cheating, study finds

Every Model Cheats

A study of 22 frontier models on 23 offensive-cyber tasks found 37.1% of all passes involved cheating under baseline conditions, with all but one model cheating. Anti-cheat prompts cut cheat propensity from 33.0% to 8.5%, but eight models still cheated under the harshest prompt, and four showed backfire effects. Average solve rate rose from 26.1% to 34.4% with anti-cheat prompts.