一句话：通过自动生成并跨 JVM 执行测试，检测 Java Reflection API 的 Javadoc 欠定规范与实现不一致。
本文研究 Java 反射 API 的 Javadoc 规范与 JVM 实现的一致性。自然语言规范使欠定规范与实现不一致难以系统发现，现有测试生成器也难以检测。作者分析两个主流 JVM 测试套件，并调查 130 名开发者；提出自动生成测试用例、跨 JVM 执行，再人工确认候选的技术。在 439 个程序上评估，该技术在 7 个类的 32 个公共方法中发现候选：12 个欠定规范候选，3 个被接受（25%）；向 Eclipse OpenJ9 报告 24 个不一致候选，21 个被接受并修复（87.5%）；向 Oracle JVM 报告 7 个，5 个被接受、4 个被修复。
Developers of widely used Java Virtual Machines (JVMs) implement and test the Java Reflection API based on a Javadoc, which is specified using a natural language. However, there is limited knowledge on whether Java Reflection API developers are able to systematically reveal i) underdetermined specifications; and ii) non-conformances between their implementation and the Javadoc. Moreover, current automatic test suite generators cannot be used to detect them. To better understand the problem, we analyze test suites of two widely used JVMs, and we conduct a survey with 130 developers who use the Java Reflection API to see whether the Javadoc impacts on their understanding. We also propose a technique to detect underdetermined specifications and non-conformances between the Javadoc and the implementations of the Java Reflection API. It automatically creates test cases, and executes them using different JVMs. Then, we manually execute some steps to identify underdetermined specifications and to confirm whether a non-conformance candidate is indeed a bug. We evaluate our technique in 439 input programs. Our technique identifies underdetermined specification and non-conformance candidates in 32 Java Reflection API public methods of 7 classes. We report underdetermined specification candidates in 12 Java Reflection API methods. Java Reflection API specifiers accept 3 underdetermined specification candidates (25%). We also report 24 non-conformance candidates to Eclipse OpenJ9 JVM, and 7 to Oracle JVM. Eclipse OpenJ9 JVM developers accept and fix 21 candidates (87.5%), and Oracle JVM developers accept 5 and fix 4 non-conformance candidates.
DOI 原文 ·
@inproceedings{paperbot5074,
title = {Java reflection API: revealing the dark side of the mirror},
author = {Felipe Pontes and Rohit Gheyi and Sabrina Souto and Alessandro Garcia and Márcio Ribeiro},
booktitle = {FSE},
year = {2019},
doi = {10.1145/3338906.3338946}
}