一句话：系统比较 CI 回归测试中学习排序与强化学习两类测试用例优先级技术。
持续集成中回归测试受提交间隔限制，需在无法全量运行的超大测试套件内选择或排序测试用例。论文将基于机器学习的测试用例优先级策略分为 learning-to-rank 与 ranking-to-learn（即强化学习）两类，为 CI 实践引入十种算法，并使用 Apache Commons 项目中的研究对象进行综合对比。研究还分析了被测代码特征与测试过程特征的影响，最终给出选择与调优最适合特定上下文技术的准则，以支持测试人员决策。
In Continuous Integration (CI), regression testing is constrained by the time between commits. This demands for careful selection and/or prioritization of test cases within test suites too large to be run entirely. To this aim, some Machine Learning (ML) techniques have been proposed, as an alternative to deterministic approaches. Two broad strategies for ML-based prioritization are learning-to-rank and what we call ranking-to-learn (i.e., reinforcement learning). Various ML algorithms can be applied in each strategy. In this paper we introduce ten of such algorithms for adoption in CI practices, and perform a comprehensive study comparing them against each other using subjects from the Apache Commons project. We analyze the influence of several features of the code under test and of the test process. The results allow to draw criteria to support testers in selecting and tuning the technique that best fits their context.
DOI 原文 ·
@inproceedings{paperbot5586,
title = {Learning-to-rank vs ranking-to-learn},
author = {Antonia Bertolino and Antonio Guerriero and Breno Miranda and Roberto Pietrantuono and Stefano Russo},
booktitle = {ICSE},
year = {2020},
doi = {10.1145/3377811.3380369}
}