一句话：通过分析测试隔离进程间通信机制，定位并修复 Maven 测试执行中的性能缺陷，使单测试平均提速约 350 毫秒。
可靠测试常通过将每个测试作为独立进程运行来避免测试顺序依赖和共享状态污染，但这会带来显著开销。本文是经验报告，旨在理解这些开销的来源，并构建系统以确认可能的最小开销。作者发现不同构建系统使用不同的进程间通信机制，这一设计选择会导致测试执行速度差异。由此他们发现 Apache Maven 测试运行代码中存在显著性能缺陷：与 Ant 相比，每个测试平均慢约 350 毫秒。该缺陷在真实项目中会显著增加测试时间。作者提交的补丁已被 Maven 集成，并继续改进 Maven 的测试执行工具。
Testing is the most common approach used in industry for checking software correctness. Developers frequently practice reliable testing-executing individual tests in isolation from each other-to avoid test failures caused by test-order dependencies and shared state pollution (e.g., when tests mutate static fields). A common way of doing this is by running each test as a separate process. Unfortunately, this is known to introduce substantial overhead. This experience report describes our efforts to better understand the sources of this overhead and to create a system to confirm the minimal overhead possible. We found that different build systems use different mechanisms for communicating between these multiple processes, and that because of this design decision, running tests with some build systems could be faster than with others. Through this inquiry we discovered a significant performance bug in Apache Maven’s test running code, which slowed down test execution by on average 350 milliseconds per-test when compared to a competing build system, Ant. When used for testing real projects, this can result in a significant reduction in testing time. We submitted a patch for this bug which has been integrated into the Apache Maven build system, and describe our ongoing efforts to improve Maven’s test execution tooling.
DOI 原文 ·
@inproceedings{paperbot6036,
title = {Debugging the performance of Maven’s test isolation: experience report},
author = {Pengyu Nie and Ahmet Celik and Matthew Coley and Aleksandar Milicevic and Jonathan Bell and Milos Gligoric},
booktitle = {ISSTA},
year = {2020},
doi = {10.1145/3395363.3397381}
}