一句话：系统评估七种推理时防御并提出包效用指标，发现 RAG 与 Self-Refine 在对抗提示下更稳健。
LLM 生成代码时常幻觉出不存在的软件包，形成供应链攻击入口。本文指出既有评测会把标准库模块误判为幻觉，使幻觉率虚高，Python 上达 9.4 个百分点。作者在 8 个模型、5 个家族和 4 种语言上评估七种推理时防御：五种引导解码、Self-Refine 与 RAG，并引入包效用衡量推荐有效性。RAG 在 32 个模型-语言配置中的 18 个降低包幻觉率，Greedy 的平均缓解-效用权衡最佳。对抗提示下幻觉率最高上升 45 个百分点，Ruby 最脆弱（80.9–95.2%），此时 RAG 与 Self-Refine 优于纯解码策略。
LLMs are increasingly used for code generation, yet they frequently hallucinate non-existent software packages, creating exploitable entry points into the software supply chain. We make four contributions to this problem. First, we show that prior evaluation methodologies systematically inflate hallucination rates by misclassifying standard-library modules as hallucinations in some languages. For Python, the overestimation reaches 9.4 percentage points. Second, we evaluate seven inference-time defenses for mitigating package hallucinations, including five guided decoding strategies (Greedy, Contrastive, DoLa, Nudging, and Active Layer-Contrastive Decoding), an iterative self-refinement approach (Self-Refine), and a Retrieval-Augmented Generation (RAG)-based defense. Across eight models spanning five families and four programming languages (Python, JavaScript, Ruby, Rust), RAG reduces the package hallucination rate (PHR) in 18 of 32 model–language configurations. Third, we introduce Package Utility (PU) to assess whether defenses preserve valid and task-relevant recommendations. Among strategies evaluated, Greedy decoding provides the strongest average mitigation–utility trade-off. Fourth, we stress-test all strategies under adversarial prompts seeded with fabricated package names and find that PHR surges by up to 45 percentage points relative to standard prompts, with Ruby consistently the most vulnerable language (80.9–95.2%). Under adversarial conditions, RAG and Self-Refine outperform all decoding-only strategies, indicating that robust defense requires either external grounding or iterative self-verification when prompts are actively hostile. Our results recast package hallucination as both a measurement problem and a decoding-time control problem, and they demonstrate that the choice of defense must be matched to the threat model and recommendation utility.
@inproceedings{paperbot12394,
title = {Evaluating Inference-Time Defenses against Package Hallucination in LLM-Generated Code},
author = {Alberick Euraste Djire and Iyiola E. Olatunji and Melissa Tessa and Earl T. Barr and Jacques Klein and Tegawendé F. Bissyandé},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3837555}
}