一句话：提出细粒度方法 Tscope，将自然语言测试用例分解为测试实体与关系元组，以精准检测冗余用例。
自然语言测试用例在系统级测试中常见，但需求冗余、并行测试和人员更替导致大量冗余用例，显著增加成本。现有方法通常将文本视为整体比较相似度，精度较低。作者观察到用例包含受测组件、约束等面向测试的实体及其关系，遂定义五类测试实体和四类关联关系，将冗余检测重构为基于实体与关系的细粒度测试内容比较。提出 Tscope，将用例解剖为原子测试元组，并设计上下文感知模型自动抽取实体和关系。在来自十个项目的 3,467 条测试用例上评估，取得 91.8% 精确率、74.8% 召回率和 82.4% F1，显著优于已有先进方法和常用分类器。
Natural language (NL) documentation is the bridge between software managers and testers, and NL test cases are prevalent in system-level testing and other quality assurance activities. Due to reasons such as requirements redundancy, parallel testing, tester turn-over within long evolving history, there are inevitably lots of redundant test cases, which significantly increase the cost. Previous redundancy detection approaches typically treat the textual descriptions as a whole to compare their similarity and suffer from low precision. Our observation reveals that a test case can have explicit test-oriented entities, such as tested function Components, Constraints, etc; and there are also specific relations between these entities. This inspires us with a potential opportunity for accurate redundancy detection. In this paper, we first define five test-oriented entity categories and four associated relation categories, and re-formulate the NL test case redundancy detection problem as the comparison of detailed testing content guided by the test-oriented entities and relations. Following that, we propose Tscope, a fine-grained approach for redundant NL test case detection by dissecting test cases into atomic test tuple(s) with the entities restricted by associated relations. To serve as the test case dissection, Tscope designs a context-aware model for the automatic entity and relation extraction. Evaluation on 3,467 test cases from ten projects shows Tscope could achieve 91.8% precision, 74.8% recall and 82.4% F1, significantly outperforming state-of-the-art approaches and commonly-used classifiers. This new formulation of the NL test case redundant detection problem can motivate the follow-up studies in further improving this task and other related tasks involving NL descriptions.
@inproceedings{paperbot7017,
title = {Putting them under microscope: a fine-grained approach for detecting redundant test cases in natural language},
author = {Zhiyuan Chang and Mingyang Li and Junjie Wang and Qing Wang and Shoubin Li},
booktitle = {FSE},
year = {2022},
doi = {10.1145/3540250.3549089}
}