一句话:用代码表示向量化对程序素材聚类去冗,配合反馈驱动选择与增强测试预言,显著提升基于合成的 JVM 测试缺陷发现能力。
基于合成的 JVM 测试通过拼装代码片段(程序素材)构造测试程序,已被证明是当前最优的 JVM 测试方法之一。已有工作主要关注保证所合成测试程序的合法性,却忽视了素材空间过于庞大这一瓶颈,导致素材探索效率与测试性能受限。本文提出 VECT(Vectorized JVM Testing):其核心思想是借助最先进的代码表示技术将程序素材向量化,并通过聚类语义相近的素材来压缩巨大的素材空间。为使方法完整且更有效,VECT 在向量化素材的基础上进一步设计了反馈驱动的素材选择策略与增强的测试预言。作者在 HotSpot、OpenJ9 和 Bisheng JDK 三款主流 JVM 上、涵盖五个 OpenJDK 版本进行了大规模实验。结果表明,在相同测试时间内,VECT 发现的唯一不一致性比当前最优技术多 115.03%~776.92%,并检出 26 个此前未知的缺陷,其中 15 个已被开发者确认或修复。
JVM testing is one of the most widely-used methodologies for guaranteeing the quality of JVMs. Among various JVM testing techniques, synthesis-based JVM testing, which constructs a test program by synthesizing various code snippets (also called program ingredients), has been demonstrated state-of-the-art. The existing synthesis-based JVM testing work puts more efforts in ensuring the validity of synthesized test programs, but ignores the influence of huge ingredient space, which largely limits the ingredient exploration efficiency as well as JVM testing performance. In this work, we propose Vectorized JVM Testing (called VECT) to further promote the performance of synthesis-based JVM testing. Its key insight is to reduce the huge ingredient space by clustering semantically similar ingredients via vectorizing ingredients using state-of-the-art code representation. To make VECT complete and more effective, based on vectorized ingredients, VECT further designs a feedback-driven ingredient selection strategy and an enhanced test oracle. We conducted an extensive study to evaluate VECT on three popular JVMs (i.e., HotSpot, OpenJ9, and Bisheng JDK) involving five OpenJDK versions. The results demonstrate VECT detects 115.03% ~ 776.92% more unique inconsistencies than the state-of-the-art JVM testing technique during the same testing time. In particular, VECT detects 26 previously unknown bugs for them, 15 of which have already been confirmed/fixed by developers.