一句话：提出ProxyWar，将LLM生成代码作为游戏智能体，通过多智能体锦标赛动态评估其代码生成质量。
现有LLM代码生成评估受限于静态基准和简单指标，难以反映程序在真实动态环境中的运行效果。ProxyWar将LLM生成的智能体嵌入多种竞技游戏环境，结合自动化测试、迭代式代码修复和多智能体锦标赛，不仅评估功能正确性，还考察生成程序的实际运行特性。在多种先进代码生成模型和游戏上的实验表明，基准分数与动态环境中的真实表现存在显著差异，揭示了此前被忽视的局限与改进机会。该工作呼吁采用更丰富的、基于竞争的代码生成评估方法，并为LLM驱动的算法发现、自适应问题求解以及实际效率与鲁棒性研究奠定基础，包括模型超越手工设计智能体的可能性。
Large language models (LLMs) have revolutionized automated code generation, yet the evaluation of their real-world effectiveness remains limited by static benchmarks and simplistic metrics. We present ProxyWar, a novel framework that systematically assesses code generation quality by embedding LLM-generated agents within diverse, competitive game environments. Unlike existing approaches, ProxyWar evaluates not only functional correctness but also the operational characteristics of generated programs, combining automated testing, iterative code repair, and multi-agent tournaments to provide a holistic view of program behavior. Applied to a range of state-of-the-art coders and games, our approach uncovers notable discrepancies between benchmark scores and actual performance in dynamic settings, revealing overlooked limitations and opportunities for improvement. These findings highlight the need for richer, competition-based evaluation of code generation. Looking forward, ProxyWar lays a foundation for research into LLM-driven algorithm discovery, adaptive problem solving, and the study of practical efficiency and robustness, including the potential for models to outperform hand-crafted agents.