一句话：提出 PerfCI 工具链，支持在 CI 中自动开展性能测试并反馈代码性能问题。
软件性能测试是识别优化机会的重要质量保证手段，但在持续集成（CI）中完全自动化运行并提供可操作反馈缺乏工具支持，因而常被排除在 CI 范围之外。本文提出工具链 PerfCI，旨在让开发者能够轻松设置并执行 CI 下的自动化性能测试。PerfCI 允许用户（1）指定性能测试任务，（2）通过扩展性能分析框架 VyPR，对从简单脚本到完整 Flask Web 服务的多种 Python 项目进行单元测试分析，（3）评估性能数据并生成针对代码的反馈。作者通过在 CERN 紧凑缪子线圈（CMS）实验中的 Web 服务上使用该工具链，展示了其可行性。
Software performance testing is an essential quality assurance mechanism that can identify optimization opportunities. Automating this process requires strong tool support, especially in the case of Continuous Integration (CI) where tests need to run completely automatically and it is desirable to provide developers with actionable feedback. A lack of existing tools means that performance testing is normally left out of the scope of CI. In this paper, we propose a toolchain - PerfCI - to pave the way for developers to easily set up and carry out automated performance testing under CI. Our toolchain is based on allowing users to (1) specify performance testing tasks, (2) analyze unit tests on a variety of python projects ranging from scripts to full-blown flask-based web services, by extending a performance analysis framework (VyPR) and (3) evaluate performance data to get feedback on the code. We demonstrate the feasibility of our toolchain by using it on a web service running at the Compact Muon Solenoid (CMS) experiment at the world's largest particle physics laboratory --- CERN.
DOI 原文 ·
@inproceedings{paperbot5951,
title = {PerfCI},
author = {Omar Javed and Joshua Heneage Dawes and Marta Han and Giovanni Franzoni and Andreas Pfeiffer and Giles Reger and Walter Binder},
booktitle = {ASE},
year = {2020},
doi = {10.1145/3324884.3415288}
}