一句话：对100个集成LLM与RAG的开源应用开展实证研究，归纳18种集成缺陷模式并提出系统性修复指南。
论文关注LLM与检索增强生成（RAG）被集成进软件系统时面临的难题：缺乏接口规范、软件上下文需求多样、系统管理复杂。作者对100个支持RAG的LLM开源应用开展全面实证研究，归纳出18种集成缺陷模式，发现77%的应用存在三种以上会损害软件功能、效率与安全的集成缺陷。基于研究结论，论文提出覆盖软件生命周期的系统性修复指南，并构建开源缺陷库HYDRANGEA，为开发者正确集成LLM提供参考。
Large language models (LLMs) provide effective solutions in various application scenarios, with the support of retrieval-augmented generation (RAG). However, developers face challenges in integrating LLM and RAG into software systems, due to lacking interface specifications, various requirements from software context, and complicated system management. In this paper, we have conducted a comprehensive study of 100 open-source applications that incorporate LLMs with RAG support, and identified 18 defect patterns. Our study reveals that 77 % of these applications contain more than three types of integration defects that degrade software functionality, efficiency, and security. Guided by our study, we propose systematic guidelines for resolving these defects in software life cycle. We also construct an open-source defect library HYDRANGEA [1].
DOI 原文 ·
@inproceedings{paperbot8931,
title = {Are LLMs Correctly Integrated into Software Systems?},
author = {Yuchen Shao and Yuheng Huang and Jiawei Shen and Lei Ma and Ting Su and Chengcheng Wan},
booktitle = {ICSE},
year = {2025},
doi = {10.1109/icse55347.2025.00204}
}