一句话：提出首个历史驱动的测试程序合成技术JavaTailor，通过复用历史缺陷代码片段生成多样测试程序以暴露JVM缺陷。
JVM 是 Java 程序正确运行的运行时环境，因此保障 JVM 实现本身的正确性与鲁棒性至关重要。现有工作通过生成可能触发缺陷的测试程序来暴露 JVM 缺陷，但多依赖细微的语法/语义变异，生成程序的多样性与有效性不足。本文提出 JavaTailor，首个历史驱动的测试程序合成技术：它先从历史缺陷触发测试程序中抽取五类代码片段，再迭代地将其插入种子程序，并通过引入额外数据依赖强化片段间交互，从而生成覆盖更多 JVM 行为/路径的测试程序；最后用这些程序对 JVM 进行差分测试。在 HotSpot 与 OpenJ9 上的实验表明，JavaTailor 生成的测试程序在代码覆盖率与检出的不一致性数量上均优于当前最佳技术，并发现 10 个未知缺陷，其中 6 个已被确认或修复。
Java Virtual Machine (JVM) provides the runtime environment for Java programs, which allows Java to be "write once, run anywhere". JVM plays a decisive role in the correctness of all Java programs running on it. Therefore, ensuring the correctness and robustness of JVM implementations is essential for Java programs. To date, various techniques have been proposed to expose JVM bugs via generating potential bug-revealing test programs. However, the diversity and effectiveness of test programs generated by existing research are far from enough since they mainly focus on minor syntactic/semantic mutations. In this paper, we propose JavaTailor, the first history-driven test program synthesis technique, which synthesizes diverse test programs by weaving the ingredients extracted from JVM historical bug-revealing test programs into seed programs for covering more JVM behaviors/paths. More specifically, JavaTailor first extracts five types of code ingredients from the historical bug-revealing test programs. Then, to synthesize diverse test programs, it iteratively inserts the extracted ingredients into the seed programs and strengthens their interactions via introducing extra data dependencies between them. Finally, JavaTailor employs these synthesized test programs to differentially test JVMs. Our experimental results on popular JVM implementations (i.e., HotSpot and OpenJ9) show that JavaTailor outperforms the state-of-the-art technique in generating more diverse and effective test programs, e.g., test programs generated by JavaTailor can achieve higher JVM code coverage and detect many more unique inconsistencies than the state-of-the-art technique. Furthermore, JavaTailor has detected 10 previously unknown bugs, 6 of which have been confirmed/fixed by developers.
@inproceedings{ZhaoWCLWZZ22,
title = {History-driven test program synthesis for JVM testing},
author = {Yingquan Zhao and Zan Wang and Junjie Chen and Mengdi Liu and Mingyuan Wu and Yuqun Zhang and Lingming Zhang},
booktitle = {ICSE},
year = {2022},
doi = {10.1145/3510003.3510059}
}