一句话：将传统测试用例优先级排序（TCP）应用于配置测试（ctest），评估84种技术并显著加速误配置检测。
配置变更是大规模软件系统部署失败的主要诱因之一，现代云系统中配置变更每日可达数百至数千次，因此检查配置变更对预防误配置故障至关重要。已有工作提出配置测试技术 Ctest，它针对使用被变更配置的代码进行测试，能自动生成大量可有效检测误配置（包括传统技术难以检测的误配置）的 ctest；然而运行 ctest 检测误配置耗时很长。受传统测试用例优先级排序（TCP）通过重排测试执行以加速回归故障检测的启发，本文提出将 TCP 用于重排 ctest，以加速误配置检测。作者广泛评估了共84种传统及面向 ctest 的新型 TCP 技术，并在五个广泛使用的云项目上进行实验。结果表明 TCP 能够显著加快误配置检测，并为在实践中将 TCP 应用于配置测试提供了指导。
Configuration changes are among the dominant causes of failures of large-scale software system deployment. Given the velocity of configuration changes, typically at the scale of hundreds to thousands of times daily in modern cloud systems, checking these configuration changes is critical to prevent failures due to misconfigurations. Recent work has proposed configuration testing, Ctest, a technique that tests configuration changes together with the code that uses the changed configurations. Ctest can automatically generate a large number of ctests that can effectively detect misconfigurations, including those that are hard to detect by traditional techniques. However, running ctests can take a long time to detect misconfigurations. Inspired by traditional test-case prioritization (TCP) that aims to reorder test executions to speed up detection of regression code faults, we propose to apply TCP to reorder ctests to speed up detection of misconfigurations. We extensively evaluate a total of 84 traditional and novel ctest-specific TCP techniques. The experimental results on five widely used cloud projects demonstrate that TCP can substantially speed up misconfiguration detection. Our study provides guidelines for applying TCP to configuration testing in practice.
DOI 原文 ·
@inproceedings{paperbot6679,
title = {Test-case prioritization for configuration testing},
author = {Runxiang Cheng and Lingming Zhang and Darko Marinov and Tianyin Xu},
booktitle = {ISSTA},
year = {2021},
doi = {10.1145/3460319.3464810}
}