一句话：提出 CoReEval 基准，系统考察提示策略、解码参数与人格框架对 LLM 代码可读性评估与人类判断对齐度的影响。
代码可读性对软件维护至关重要，却难以大规模评估：传统度量无法捕捉主观的人类判断，而人工评估昂贵且不一致。本文提出 CoReEval 基准，覆盖 10 个 LLM 与 656 个代码片段（Java、Python、CUDA）上的 140 万次评估，并系统研究提示策略、解码参数以及人格（persona）框架如何影响 LLM 与人类判断的对齐程度。结果显示，LLM 在 Java 上呈现中等对齐（r=0.25），而在 Python 上接近零（r=0.02）、在 CUDA 上为零（r=0.00）；开发者引导式提示可显著提升方面覆盖度（Java 测试中的 Structure 从 15% 提升到 94%），但也会增大分数方差。作者认为，当配置以人为中心的评价标准时，LLM 有望成为自动化代码审查、开发者教育与 CI/CD 中可解释代码可读性评估的有前景工具。
Code readability is essential for software maintenance, yet difficult to assess at scale. Traditional metrics fail to capture subjective human judgments, while manual evaluation is expensive and inconsistent. We introduce CoReEval, a benchmark comprising 1.4 million evaluations across 10 LLMs and 656 code snippets (Java, Python, CUDA). We investigate how prompting strategies, decoding parameters, and persona framing influence LLM alignment with human judgments. LLMs exhibit moderate alignment in Java (r=0.25) but near-zero in Python (r=0.02) and CUDA (r=0.00). Developer-guided prompting dramatically improves aspect coverage (Java test Structure: 15% to 94%) but increases score variance. When configured with human-centric criteria, LLMs are promising tools for interpretable code readability assessment in automated code review, developer education, and CI/CD.
@inproceedings{paperbot11812,
title = {From Metrics to Models: Large Language Models as Human-Aligned Code Readability Evaluators},
author = {Wendkûuni C. Ouédraogo and Yinghua Li and Xueqi Dang and Paweł Borsukiewicz and Xin Zhou and Anıl Koyuncu and Jacques Klein and David Lo and Tegawendé F. Bissyandé},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3774748.3795656}
}