一句话:提出距离采样策略,按概率分布覆盖配置空间,提升性能预测模型精度。
可配置软件系统配置组合爆炸,暴力测量所有配置性能不可行。现有采样策略(如 t-wise)依赖机器学习,仅分析少量配置来预测其他配置性能,但往往无法覆盖具有显著性能差异的重要配置。本文发现多数采样策略在覆盖相关性能值方面效果不佳,据此提出基于距离的采样策略:利用距离度量和概率分布,使样本配置按给定分布散布于配置空间,从而覆盖配置选项间的不同交互。作者在 10 个真实可配置软件系统上与 t-wise 等先进采样策略进行比较,结果表明距离采样在中到大型样本集上能获得更准确的性能模型。
Configurable software systems provide a multitude of configuration options to adjust and optimize their functional and non-functional properties. For instance, to find the fastest configuration for a given setting, a brute-force strategy measures the performance of all configurations, which is typically intractable. Addressing this challenge, state-of-the-art strategies rely on machine learning, analyzing only a few configurations (i.e., a sample set) to predict the performance of other configurations. However, to obtain accurate performance predictions, a representative sample set of configurations is required. Addressing this task, different sampling strategies have been proposed, which come with different advantages (e.g., covering the configuration space systematically) and disadvantages (e.g., the need to enumerate all configurations). In our experiments, we found that most sampling strategies do not achieve a good coverage of the configuration space with respect to covering relevant performance values. That is, they miss important configurations with distinct performance behavior. Based on this observation, we devise a new sampling strategy, called distance-based sampling, that is based on a distance metric and a probability distribution to spread the configurations of the sample set according to a given probability distribution across the configuration space. This way, we cover different kinds of interactions among configuration options in the sample set. To demonstrate the merits of distance-based sampling, we compare it to state-of-the-art sampling strategies, such as t-wise sampling, on $10$ real-world configurable software systems. Our results show that distance-based sampling leads to more accurate performance models for medium to large sample sets.
@inproceedings{KalteneckerGSGA19,
title = {Distance-Based Sampling of Software Configuration Spaces},
author = {Christian Kaltenecker and Alexander Grebhahn and Norbert Siegmund and Jianmei Guo and Sven Apel},
booktitle = {ICSE},
year = {2019},
doi = {10.1109/icse.2019.00112}
}