paperbot · PL 论文追踪

全部论文 待读 收藏

P2 CoMSA: A Modeling-Driven Sampling Approach for Configuration Performance Testing

CoMSA:一种面向配置性能测试的建模驱动采样方法
ASE 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE)2023匹配度 2/10 · 该论文研究配置性能测试中的建模驱动采样,属于软件测试与性能建模领域,与用户关注的PL与AI交叉、静态分析、Agentic程序、浮点程序分析或贝叶斯程序分析无直接关联;仅同属软件工程与程序分析大方向,但研究问题和方法均不匹配。引用 6 inbox
Yuanjie Xia, Zishuo Ding, Weiyi Shang
crossref: 2026-09-12dblp: 2026-09-12

AI 速览

一句话:提出建模驱动的采样方法 CoMSA,依据性能模型预测的不确定性挑选配置样本,提升性能建模精度并降低配置性能测试成本。

高度可配置系统允许用户在不同部署环境中灵活配置,但海量配置组合与有限的测试时间、资源之间的矛盾,使配置性能测试代价高昂。用性能模型近似预测配置性能是降低测试成本的主要手段,然而已有建模与采样技术在选择训练样本时通常不考虑性能模型自身的精度,导致实际建模效果次优。本文提出建模驱动的采样方法 CoMSA,其核心直觉是:若某配置经由当前性能模型得到的预测结果不确定性越高,就越应被选为后续训练样本,以此迭代改进模型。CoMSA 同时面向两类场景设计:没有历史性能测试结果的冷启动与已有历史结果的温启动。作者在 LRZIP、LLVM、x264 和 SQLite 四个对象上开展评估,结果表明相较其他基线采样方法,CoMSA 能显著提高预测模型的精度,并提升配置性能测试的效率。

问题
高度可配置系统的配置空间规模巨大,而性能测试的时间与资源有限,如何用尽可能少的配置样本构建高精度的配置性能模型是核心难题。已有采样方法大多未考虑模型精度,导致建模结果次优。
方法
提出建模驱动的采样方法 CoMSA,利用当前性能模型对配置预测结果的不确定性来指导采样,优先选取预测最不确定的配置作为新的训练样本,并分别针对无历史测试结果的冷启动与有历史结果的温启动两种场景进行设计。
评估
在 LRZIP、LLVM、x264 和 SQLite 四个系统上,将 CoMSA 与其他基线采样方法对比,衡量其预测模型精度与配置性能测试效率。结果表明 CoMSA 能明显提升模型精度并降低测试开销。
原文摘要(Abstract)

Highly configurable systems enable customers to flexibly configure the systems in diverse deployment environments. The flexibility of configurations also poses challenges for performance testing. On one hand, there exist a massive number of possible configurations; while on the other hand, the time and resources are limited for performance testing, which is already a costly process during software development. Modeling the performance of configurations is one of the solutions to reduce the cost of configuration performance testing. Although prior research proposes various modeling and sampling techniques to build configuration performance models, the sampling approaches used in the model typically do not consider the accuracy of the performance models, leading to potential sub-optimal performance modeling results in practice. In this paper, we present a modeling-driven sampling approach (CoMSA) to improve the performance modeling of highly configurable systems. The intuition of CoMSA is to select samples based on their uncertainties to the performance models. In other words, the configurations that have the more uncertain performance prediction results by the performance models are more likely to be selected as further training samples to improve the model. CoMSA is designed by considering both scenarios where 1) the software projects do not have historical performance testing results (cold start) and 2) there exist historical performance testing results (warm start). We evaluate the performance of our approach in four subjects, namely LRZIP, LLVM, x264, and SQLite. Through the evaluation result, we can conclude that our sampling approaches could highly enhance the accuracy of the prediction models and the efficiency of configuration performance testing compared to other baseline sampling approaches.

链接与引用

DOI 原文 · DBLP

BibTeX
@inproceedings{XiaDS23,
  title = {CoMSA: A Modeling-Driven Sampling Approach for Configuration Performance Testing},
  author = {Yuanjie Xia and Zishuo Ding and Weiyi Shang},
  booktitle = {ASE},
  year = {2023},
  doi = {10.1109/ase56229.2023.00091}
}