一句话：通过定性分析 Stack Overflow 与 GitHub 数据，揭示数据科学库错误消息「糟糕」的成因与修复方式。
错误消息是调试的关键起点，却常被诟病为晦涩、令人困惑、信息量不足。本文聚焦数据科学开发这一日益重要却研究不足的领域，对 201 Stack Overflow 帖子与 335 个 GitHub issue 进行定性分析，系统考察「糟糕」错误消息的成因与修复。研究发现，其成因远不止措辞不当或表述缺陷：由于库无法知晓用户意图、也无法「看到」外部错误，许多错误消息本质上就具有误导性或无信息量，且难以避免。错误消息相关问题的修复大多涉及源代码变更，纯粹的文本内容更新只占很小比例。此外，错误消息是否有信息量、是否有帮助并非总是界限分明；即便是明确指认故障与解决方案的错误消息，也可能令部分用户困惑。作者据此呼吁未来更深入地研究错误消息的评估与改进方法。
Error messages are critical starting points for debugging. Unfortunately, they seem to be notoriously cryptic, confusing, and uninformative. Yet, it still remains a mystery why error messages receive such bad reputations, especially given that they are merely very short pieces of natural language text. In this paper, we empirically demystify the causes and fixes of "bad" error messages, by qualitatively studying 201 Stack Overflow threads and 335 GitHub issues. We specifically focus on error messages encountered in data science development, which is an increasingly important but not well studied domain. We found that the causes of "bad" error messages are far more complicated than poor phrasing or flawed articulation of error message content. Many error messages are inherently and inevitably misleading or uninformative, since libraries do not know user intentions and cannot "see" external errors. Fixes to error-message-related issues mostly involve source code changes, while exclusive message content updates only take up a small portion. In addition, whether an error message is informative or helpful is not always clear-cut; even error messages that clearly pinpoint faults and resolutions can still cause confusion for certain users. These findings thus call for a more in-depth investigation on how error messages should be evaluated and improved in the future.
DOI 原文 ·
@inproceedings{paperbot6379,
title = {Demystifying “bad” error messages in data science libraries},
author = {Yida Tao and Zhihui Chen and Yepang Liu and Jifeng Xuan and Zhiwu Xu and Shengchao Qin},
booktitle = {FSE},
year = {2021},
doi = {10.1145/3468264.3468560}
}