一句话：提出 LLM4JMH，用 LLM 从源码生成 Java 性能微基准，并以静态分析增强，实现早期性能回归检测。
性能回归威胁软件可靠性，而编写性能微基准成本高、易错且依赖专家经验。现有方法多从功能单元测试生成微基准，本文提出基于 LLM 的方法 LLM4JMH，研究能否直接从 Java 源码生成可靠有效的性能微基准，并探索静态分析等程序分析技术对生成质量的影响。实验在 RxJava、Eclipse Collections、Zipkin 上评估：LLM 生成的微基准的缺陷检测效果可与专家手写微基准相当，并将整体预期检测延迟最多降低 52.60%；在 Apache Flink 的 11 个真实性能缺陷中成功识别 6 个。结果表明该方法可支持 CI 流水线中的早期性能回归检测，减少对专家测试的依赖。
Performance regressions pose a significant threat to software reliability. In recent years, smaller-scale performance testing, such as performance microbenchmarks, has been adopted in practice to detect such regressions in an early stage of development. Developing these microbenchmarks is costly, error-prone, and demands specialized expertise. Motivated by the recent progress of large language models (LLMs) in code-related tasks, we study whether LLMs can step in as performance experts to automate the generation of microbenchmarks for performance regression detection. While existing approaches typically depend on functional unit tests to generate performance microbenchmarks, we instead design an LLM-based approach, LLM4JMH, to assess the capability of LLMs in generating reliable and effective performance microbenchmarks directly from source code. We further explore how program analysis techniques, such as static analysis, can enhance the reliability and effectiveness of LLM-generated performance microbenchmarks. Experimental results show that LLM-generated performance microbenchmarks achieve comparable bug detection effectiveness to expert-written microbenchmarks, while reducing overall expected detection latency by up to 52.60% across RxJava, Eclipse Collections, and Zipkin. In the study, the generated tests can successfully identify six out of 11 real-world performance bugs in Apache Flink. These results demonstrate that LLM-based performance microbenchmark generation can automate early performance regression detection in continuous integration pipelines of the software development process and reduce reliance on expert-crafted tests, advancing performance-aware software engineering.
@inproceedings{paperbot11807,
title = {LLM4JMH: Studying the Use of LLMs for Generating Java Performance Microbenchmarks},
author = {Zongxiong Chen and Derui Zhu and Kundi Yao and Weiyi Shang and Jinfu Chen and Jiahui Geng and Alexander Pretschner and Jens Grossklags and Manfred Hauswirth and Sonja Schimmler},
booktitle = {ICSE},
year = {2026},
doi = {10.1145/3744916.3787777}
}