一句话：实证研究显示初学者理解 LLM 生成代码的单任务成功率仅 32.5%，代码理解是 LLM 辅助编程的关键瓶颈。
LLM 正被越来越多地用于编程工作，已有研究关注提示技巧，但生成后的代码必须被阅读、评估和整合或拒绝，初学者能否胜任这些任务尚不清楚。本文对 32 名 CS1 学生、160 个任务实例开展对照研究，比较他们对自然语言函数描述与 LLM 生成实现的理解，并探究判断代码正确性时的困难。结果显示单任务成功率仅 32.5%，困难在不同人口群体中普遍存在；关键障碍包括非英语母语者的语言壁垒、对 Python 语法不熟悉以及自动化偏见。研究结论表明，代码理解是初学者借助 LLM 写代码时面临的重要屏障。
Large language models (LLMs) are being increasingly adopted for programming work. Prior work shows that while LLMs accelerate task completion for professional programmers, beginning programmers struggle to prompt models effectively. However, prompting is just half of the code generation process -- when code is generated, it must be read, evaluated, and integrated (or rejected). How accessible are these tasks for beginning programmers? This paper measures how well beginners comprehend LLM-generated code and explores the challenges students face in judging code correctness. We compare how well students understand natural language descriptions of functions and LLM-generated implementations, studying 32 CS1 students on 160 task instances. Our results show a low per-task success rate of 32.5\%, with indiscriminate struggles across demographic populations. Key challenges include barriers for non-native English speakers, unfamiliarity with Python syntax, and automation bias. Our findings highlight the barrier that code comprehension presents to beginning programmers seeking to write code with LLMs.
@inproceedings{paperbot9002,
title = {``I Would Have Written My Code Differently': Beginners Struggle to Understand LLM-Generated Code},
author = {Yangtian Zi and Luisa Li and Arjun Guha and Carolyn Anderson and Molly Q Feldman},
booktitle = {FSE},
year = {2025},
doi = {10.1145/3696630.3731663}
}