一句话：通过EEG与LLM困惑度对比，发现人类与LLM对代码的困惑在位置和强度上相关。
论文关注人类程序员与基于大语言模型的编程助手在代码理解上是否会被同类代码困扰，因为这种不一致可能导致误解、低效与缺陷。作者开展实证研究，让LLM与人类程序员阅读清晰代码和令人困惑代码；用LLM困惑度操作化LLM的理解困难，用基于EEG的注视相关电位操作化人类困惑。结果发现，LLM困惑度峰值在位置和幅度上与人类神经生理响应相关，表明二者对代码的困惑具有相似性。基于此，作者提出一种数据驱动、基于LLM的方法来识别会引发人类程序员困惑的代码区域。
Already today, humans and programming assistants based on large language models (LLMs) collaborate in everyday programming tasks. Clearly, a misalignment between how LLMs and programmers comprehend code can lead to misunderstandings, inefficiencies, low code quality, and bugs. A key question in this space is whether humans and LLMs are confused by the same kind of code. This would not only guide our choices of integrating LLMs in software engineering workflows but also inform about possible improvements of LLMs. To this end, we conducted an empirical study comparing an LLM to human programmers comprehending clean and confusing code. We operationalized comprehension for the LLM by using LLM perplexity, and for human programmers using neurophysiological responses (in particular, EEG-based fixation-related potentials). We found that LLM perplexity spikes correlate, both in terms of location and amplitude, with human neurophysiological responses that indicate confusion. This result suggests that LLMs and humans are similarly confused about the code. Based on these findings, we devised a data-driven, LLM-based approach to identify regions of confusion in code that elicit confusion in human programmers.
@inproceedings{paperbot11559,
title = {Are Humans and LLMs Confused by the Same Code? An Empirical Study on Fixation-Related Potentials and LLM Perplexity},
author = {Youssef Abdelsalam and Norman Peitek and Anna-Maria Maurer and Mariya Toneva and Sven Apel},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3773219}
}