一句话：提出 Ceprot，用 Transformer 分两阶段自动识别随生产代码变更而过期的测试用例，并生成更新后的测试用例。
在软件演化中，测试用例需与生产代码共同演化，但受项目进度等因素影响常出现滞后，过期测试用例会提高维护成本、难以揭示缺陷甚至引入新 bug。本文提出 Ceprot（Co-Evolution of Production-Test Code），一种基于 Transformer 的两阶段方法：给定方法级的生产代码变更与对应测试用例，先判定该测试是否已过期、需要更新，若判定为过期则自动生成更新后的测试版本。作者为此构建了两个数据集。实验显示，过期识别阶段的精确率与召回率分别为 98.3% 和 90.0%；在识别为过期的样本中，12.3% 的生成测试与人工真值完全一致。动态评估表明 48.0% 的更新测试可编译，平均覆盖率达 34.2%，相对过期测试提升约 89%，并辅以人工评估验证有效性。
Software testing is one of the most essential parts of the software lifecycle and requires a substantial amount of time and effort. During the software evolution, test cases should co-evolve with the production code. However, the co-evolution of test cases often fails due to tight project schedules and other reasons. Obsolete test cases improve the cost of software maintenance and may fail to reveal faults and even lead to future bugs. Therefore, it is essential to detect and update these obsolete test cases in time. In this paper, we propose a novel approach Ceprot (Co-Evolution of Production-Test Code) to identify outdated test cases and update them automatically according to changes in the production code. Ceprot consists of two stages, i.e., obsolete test identification and updating. Specifically, given a production code change and a corresponding test case, Ceprot first identifies whether the test case should be updated. If the test is identified as obsolete, Ceprot will update it to a new version of test case. To evaluate the effectiveness of the two stages, we construct two datasets. Our dataset focuses on method-level production code changes and updates on their obsolete test cases. The experimental results show that Ceprot can effectively identify obsolete test cases with precision and recall of 98.3% and 90.0%, respectively. In addition, test cases generated by Ceprot are identical to the ground truth for 12.3% of samples that are identified as obsolete by Ceprot. We also conduct dynamic evaluation and human evaluation to measure the effectiveness of the updated test cases by Ceprot. 48.0% of updated test cases can be compiled and the average coverage of updated cases is 34.2% which achieves 89% coverage improvement over the obsolete tests. We believe that this study can motivate the co-evolution of production and test code.
@inproceedings{HuLXLXY23,
title = {Identify and Update Test Cases When Production Code Changes: A Transformer-Based Approach},
author = {Xing Hu and Zhuang Liu and Xin Xia and Zhongxin Liu and Tongtong Xu and Xiaohu Yang},
booktitle = {ASE},
year = {2023},
doi = {10.1109/ase56229.2023.00165}
}