ProgramBench Vetted:从二进制逆向重构代码
ProgramBench Vetted: Reverse Engineering from a Runnable Binary

ProgramBench Vetted 发布了一套包含 50 个任务的基准测试,旨在评估 AI 代理能否仅凭可运行二进制文件和文档,逆向重构出原始代码。与原版不同,新版本通过引入“代理法官”和“代理医生”机制,严格过滤测试重复、环境漏洞及评分捷径,确保每个奖励点都真实反映行为能力。这项挑战要求代理在长周期内维护并修正程序模型,不仅考验代码生成能力,更是对长上下文协调与强化学习奖励机制的终极试炼。
Prior exposure does not imply that a model can retrieve or reconstruct the implementation from memory when asked to move from observed behavior back to code.