一句话：分析工业 LLM 单测生成的两类幻觉，提出 DEHALL 结合上下文构建与静态分析修复，显著提升覆盖率与缺陷检测。
单元测试对工业软件质量至关重要，但 LLM 生成的测试存在编译失败与幻觉，阻碍其实际部署。论文分析蚂蚁集团生产系统中 LLM 单测的编译失败，将其归纳为上下文不足导致的外在幻觉和模型固有局限导致的内在幻觉。为此提出 DEHALL 端到端工具：构建异构图捕获相关上下文，并针对 import、字段和方法问题实施静态分析驱动的定向修复。在蚂蚁内部数据集上，DEHALL 达到 71.56% 行覆盖率和 67.18% 分支覆盖率，显著优于原始 LLM；在公开基准上，覆盖率与缺陷检测能力也优于此前最优方法。该工具已在蚂蚁多个业务域部署，开发者采用率达 81%，用户反馈积极。
Unit testing plays a critical role in ensuring software quality and reliability in large-scale industrial environments. While Large Language Models (LLMs) offer promising automated test generation capabilities, their practical deployment faces significant challenges due to hallucination problems. In this paper, we analyze compilation failures from LLM-generated unit tests in Ant Group's production systems and identify two fundamental types of hallucinations: extrinsic hallucinations caused by insufficient contextual information and intrinsic hallucinations stemming from model limitations even with adequate context. To address these issues, we propose DEHALL, an automated end-to-end unit test generation tool that systematically mitigates both types of hallucinations through comprehensive context construction and targeted static analysis-based repair. Our approach builds a heterogeneous graph to capture relevant context and employs specialized repair mechanisms for import, field, and method issues. Evaluation on Ant Group's internal datasets reveals that DEHALL achieves 71.56% line coverage and 67.18% branch coverage, significantly outperforming vanilla LLM approaches. In the public benchmarks, it also shows better performance on coverage and better defect detection capability than previous state-of-the-art approaches. DEHALL has been successfully deployed across multiple business domains at Ant Group, achieving an 81% developer adoption rate with positive user feedback on productivity improvements.
@inproceedings{TuZXSFZ26,
title = {Understanding and Mitigating Hallucinations in Industrial LLM-based Unit Test Generation},
author = {Yanlun Tu and Ziyue Zhou and Cheng Xu and Jingling Sun and Shuai Feng and Chengyu Zhang},
booktitle = {FSE},
year = {2026},
doi = {10.1145/3803437.3805260}
}