一句话：提出 PAF，依据失败执行时数据流的相似性聚类自动生成测试的失败，并按违反隐式前置条件的可能性排序，以区分真正揭示缺陷的失败与误报。
自动化单元测试工具（如 Randoop）可通过生成失败测试来发现缺陷，但其产生的失败常是误报，因而难以在实践中推广。误报的主要来源是生成的失败测试违反了被测方法的隐式前置条件（例如方法入口处某字段不应为 null），这类条件未被显式编写或文档化，却被开发者默认假定。为此，作者提出 PAF 技术：对由同一原因导致的生成失败进行聚类，并根据其违反被测方法隐式前置条件的可能性重新排序。PAF 从多次测试执行中观察失败时所用变量值的数据流，基于数据流相似性及这些值的来源对失败聚类，进而判定其揭示缺陷的可能性。作者将 PAF 集成到 Randoop 中，并在开源项目上开展实证评估，结果表明 PAF 能有效聚类由同一缺陷引起的缺陷揭示性测试，并成功将真正揭示缺陷的失败排在前面。
Automated unit testing tools, such as Randoop, have been developed to produce failing tests as means of finding faults. However, these tools often produce false alarms, so are not widely used in practice. The main reason for a false alarm is that the generated failing test violates an implicit precondition of the method under test, such as a field should not be null at the entry of the method. This condition is not explicitly programmed or documented but implicitly assumed by developers. To address this limitation, we propose a technique called PAF to cluster generated test failures due to the same cause and reorder them based on their likelihood of violating an implicit precondition of the method under test. From various test executions, PAF observes their dataflows to the variables whose values are used when the program fails. Based on the dataflow similarity and where these values are originated, PAF clusters failures and determines their likelihood of being fault revealing. We integrated PAF into Randoop. Our empirical results on open-source projects show that PAF effectively clusters fault revealing tests arising from the same fault and successfully prioritizes the fault-revealing ones.
DOI 原文 ·
@inproceedings{paperbot4582,
title = {Which generated test failures are fault revealing? prioritizing failures based on inferred precondition violations using PAF},
author = {Mijung Kim and Shing-Chi Cheung and Sunghun Kim},
booktitle = {FSE},
year = {2018},
doi = {10.1145/3236024.3236058}
}