一句话:利用测试与提交日志训练轻量机器学习模型预测测试结果,跳过部分测试以减少大规模 CI/CD 服务的测试时间。
大规模在线服务基于 CI/CD 持续集成与部署,开发者每天提交大量代码,导致测试成为既关键又耗时的任务。FastLane 提出数据驱动的测试最小化方法:利用历史测试日志与提交日志训练轻量级机器学习模型,预测各测试用例的结果;对预测结果可信的测试不再实际执行,从而节省测试时间与资源。作者在大型电子邮件与协作平台上评估。结果表明,该方法可减少 18.04%(接近五分之一)的测试时间,同时测试结果预测准确率达到 99.99%。
Today, we depend on numerous large-scale services for basic operations such as email. These services, built on the basis of Continuous Integration/Continuous Deployment (CI/CD) processes, are extremely dynamic: developers continuously commit code and introduce new features, functionality and fixes. Hundreds of commits may enter the code-base in a single day. Therefore one of the most time-critical, yet resource-intensive tasks towards ensuring code-quality is effectively testing such large code-bases. This paper presents FastLane, a system that performs data-driven test minimization. FastLane uses light-weight machine-learning models built upon a rich history of test and commit logs to predict test outcomes. Tests for which we predict outcomes need not be explicitly run, thereby saving us precious test-time and resources. Our evaluation on a large-scale email and collaboration platform service shows that our techniques can save 18.04%, i.e., almost a fifth of test-time while obtaining a test outcome accuracy of 99.99%.
@inproceedings{PhilipBKMN19,
title = {FastLane: Test Minimization for Rapidly Deployed Large-Scale Online Services},
author = {Adithya Abraham Philip and Ranjita Bhagwan and Rahul Kumar and Chandra Sekhar Maddila and Nachiappan Nagppan},
booktitle = {ICSE},
year = {2019},
doi = {10.1109/icse.2019.00054}
}