paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Code Coverage Criteria for Asynchronous Programs

异步程序的代码覆盖准则
FSE Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering2023匹配度 2/10 · 论文研究异步程序的测试覆盖准则与动态覆盖工具,属于软件测试和开发者工具方向,未涉及用户核心关注的 LLM/Agent 与静态分析交叉、浮点程序分析或贝叶斯程序分析。虽然与 PL/程序分析有较弱关联,但不在研究画像的核心问题域内。引用 8 inbox
Mohammad Ganji, Saba Alimadadi, Frank Tip
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话:提出面向异步程序的新型测试充分性覆盖准则,涵盖异步操作的完成、异常、反应注册与执行,并实现工具 JScope 予以验证。

异步软件行为复杂且易错,而传统代码覆盖准则无法充分反映异步操作的完成情况、交互与错误处理。本文提出一组新的测试充分性准则,用于度量:(i) 异步操作在正常与异常两种执行路径下的完成;(ii) 为处理这两种可能结果而进行的反应(reaction)注册;(iii) 测试对这些反应的执行。作者实现了 JScope——一个面向 JavaScript 应用、以 Visual Studio Code 交互式插件形式提供的自动覆盖度量工具。在 20 个 JavaScript 应用上的评估表明,新准则能补充传统准则、改进测试充分性评估;对 15 个真实 GitHub 异步相关问题的调查显示,新准则能揭示那些虽被传统准则判定为已覆盖、实则未被测试的异步错误行为;此外,一项有 12 名参与者参加的受控实验表明,JScope 在真实场景中能有效提升程序员评估测试充分性以及检测异步代码未被测试行为的能力。

问题
传统代码覆盖准则无法刻画异步操作的完成(含异常)、反应注册与执行,导致异步软件中未被测试的错误行为被误判为已覆盖。
方法
提出一组针对异步操作完成、反应注册与反应执行的新型覆盖准则,并实现自动度量工具 JScope(JavaScript,VS Code 交互式插件)。
评估
在 20 个 JavaScript 应用上度量覆盖率并与传统准则对比,分析 15 个真实 GitHub 异步相关 issue,并开展 12 名参与者的受控用户实验。
原文摘要(Abstract)

Asynchronous software often exhibits complex and error-prone behaviors that should be tested thoroughly. Code coverage has been the most popular metric to assess test suite quality. However, traditional code coverage criteria do not adequately reflect completion, interactions, and error handling of asynchronous operations. This paper proposes novel test adequacy criteria for measuring: (i) completion of asynchronous operations in terms of both successful and exceptional execution, (ii) registration of reactions for handling both possible outcomes, and (iii) execution of said reactions through tests. We implement JScope, a tool for automatically measuring coverage according to these criteria in JavaScript applications, as an interactive plug-in for Visual Studio Code. An evaluation of JScope on 20 JavaScript applications shows that the proposed criteria can help improve assessment of test adequacy, complementing traditional criteria. According to our investigation of 15 real GitHub issues concerned with asynchrony, the new criteria can help reveal faulty asynchronous behaviors that are untested yet are deemed covered by traditional coverage criteria. We also report on a controlled experiment with 12 participants to investigate the usefulness of JScope in realistic settings, demonstrating its effectiveness in improving programmers’ ability to assess test adequacy and detect untested behavior of asynchronous code.

链接与引用

DOI 原文 · PDF(开放获取) · DBLP

BibTeX
@inproceedings{GanjiAT23,
  title = {Code Coverage Criteria for Asynchronous Programs},
  author = {Mohammad Ganji and Saba Alimadadi and Frank Tip},
  booktitle = {FSE},
  year = {2023},
  doi = {10.1145/3611643.3616292}
}