paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Test Flimsiness: Characterizing Flakiness Induced by Mutation to the Code Under Test

测试脆弱性:刻画由被测代码变异引发的波动性
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 2/10 · 该论文研究变异测试中被测代码变异诱发 flaky test 的现象,属于软件测试/动态分析范畴;与用户核心关注的静态分析、LLM/Agent 改进静态分析与形式化验证、浮点程序分析、贝叶斯程序分析均无直接交集。引用 1 inbox
Owain Parry, Gregory M. Kapfhammer, Michael Hilton, Phil McMinn
crossref: 2026-09-12

AI 速览

一句话:揭示标准变异算子对被测代码的变异本身会诱发测试波动性(test flimsiness),威胁依赖确定性测试结果的变异测试等技术。

论文关注 flaky tests 对软件开发的挑战,并提出此前被忽视的 test flimsiness 现象:由对被测代码的变异所引发的测试波动性。这些变异由标准变异测试工具中的算子生成。作者指出,以往研究多量化既有 flaky tests 对变异测试的影响,但变异本身也能诱发 flakiness,构成此前被忽略的威胁。该现象不仅影响变异测试,也动摇任何依赖“测试对变异体具有确定性结果”这一假设的技术可靠性。论文通过实证方式对该现象进行刻画。

问题
变异测试等技术以测试对变异体结果确定为前提,但变异操作可能导致测试非确定失败;现有研究忽视了这种由变异诱发的 flakiness。
方法
使用标准变异测试工具中的变异算子生成被测代码的变异体,并对由此引发的测试波动性进行实证刻画。
评估
通过实证研究表征 test flimsiness 现象,揭示变异本身可诱发 flakiness 及其对变异测试和依赖确定性结果技术的威胁。
原文摘要(Abstract)

Flaky tests, which fail non-deterministically against the same version of code, pose a well-established challenge to software developers. In this paper, we characterize the overlooked phenomenon of test FLIMsiness: FLakiness Induced by Mutations to the code under test. These mutations are generated by the same operators found in standard mutation testing tools. Flimsiness has profound implications for software testing researchers. Previous studies quantified the impact of pre-existing flaky tests on mutation testing, but we reveal that mutations themselves can induce flakiness, exposing a previously neglected threat. This has serious effects beyond mutation testing, calling into question the reliability of any technique that relies on deterministic test outcomes in response to mutations.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot11583,
  title = {Test Flimsiness: Characterizing Flakiness Induced by Mutation to the Code Under Test},
  author = {Owain Parry and Gregory M. Kapfhammer and Michael Hilton and Phil McMinn},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3744916.3773125}
}