一句话:揭示标准变异算子对被测代码的变异本身会诱发测试波动性(test flimsiness),威胁依赖确定性测试结果的变异测试等技术。
论文关注 flaky tests 对软件开发的挑战,并提出此前被忽视的 test flimsiness 现象:由对被测代码的变异所引发的测试波动性。这些变异由标准变异测试工具中的算子生成。作者指出,以往研究多量化既有 flaky tests 对变异测试的影响,但变异本身也能诱发 flakiness,构成此前被忽略的威胁。该现象不仅影响变异测试,也动摇任何依赖“测试对变异体具有确定性结果”这一假设的技术可靠性。论文通过实证方式对该现象进行刻画。
Flaky tests, which fail non-deterministically against the same version of code, pose a well-established challenge to software developers. In this paper, we characterize the overlooked phenomenon of test FLIMsiness: FLakiness Induced by Mutations to the code under test. These mutations are generated by the same operators found in standard mutation testing tools. Flimsiness has profound implications for software testing researchers. Previous studies quantified the impact of pre-existing flaky tests on mutation testing, but we reveal that mutations themselves can induce flakiness, exposing a previously neglected threat. This has serious effects beyond mutation testing, calling into question the reliability of any technique that relies on deterministic test outcomes in response to mutations.
@inproceedings{paperbot11583,
title = {Test Flimsiness: Characterizing Flakiness Induced by Mutation to the Code Under Test},
author = {Owain Parry and Gregory M. Kapfhammer and Michael Hilton and Phil McMinn},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3773125}
}