一句话：提出 BenchPrism，自动生成多样化代码风格的基准变体，揭示代码风格给 LLM 代码评估带来的隐性偏差。
大语言模型在软件工程任务上快速迭代并取得性能提升，但现有主流基准大多只衡量功能正确性，忽视了代码风格等非功能属性，这类属性在真实开发中至关重要，也可能给评估引入偏差。为研究这种隐性偏差，作者提出 BenchPrism：它自动地对代码施加多样化的风格变换，并生成“风格化”的基准变体用于 LLM 评估，从而系统考察偏差在不同基准、不同 LLM 与不同代码任务上的表现及其受何种因素影响。实验证实该偏差确实存在，导致 LLM 评估性能出现不稳定波动——从下降 14.92% 到上升 7.51%——甚至会扭曲模型之间的排名。论文据此给出经验性启示，呼吁研究者在评估 LLM 时关注此类隐藏偏差带来的鲁棒性问题。
Rapid iteration of large language models (LLMs) improved their performance on software engineering tasks. While prevailing benchmarks effectively measure LLMs' functional correctness, they often overlook non-functional aspects, such as coding styles, which are crucial in real-world development and can introduce evaluation bias. To investigate such implicit bias, we propose BenchPrism that automatically disperses diverse coding styles and generates style-fulfilled benchmark variants for LLM evaluation, and thus explore how the bias behaves and is influenced across different benchmarks, LLMs, and code tasks. Experiments confirm the existence of bias, which indeed leads to unstable performance deviations in LLM evaluation (from a 14.92% drop to a 7.51% increase), even distorting model rankings. We also raise valuable takeaways and hope to inspire future researchers to be mindful of the robustness of LLM evaluation when considering the possibility of such hidden bias.
@inproceedings{LiuJW26,
title = {The Stylistic Blind Spot: Uncovering the Hidden Implicit Bias of Coding Style on LLM Code Evaluation},
author = {Zhiyuan Liu and Yingying Jiang and Huiyan Wang},
booktitle = {FSE},
year = {2026},
doi = {10.1145/3803437.3805590}
}