一句话：构建自动化工具采集 33,001 个 LLM 应用仓库，形成面向安全研究的大规模数据集。
随着大语言模型（LLM）的广泛采用，基于 LLM 的应用大量涌现，但也带来提示注入、数据泄露、未授权代码执行等安全问题。已有研究多聚焦于少数特定漏洞和少量应用，难以刻画真实世界中 LLM 应用的安全威胁全貌。为支撑该领域研究，作者开发了一个自动化工具，系统性地收集基于 LLM API 或框架的独立应用并构建数据集。该数据集包含 33,001 个代码仓库，为安全分析提供了广泛而有代表性的基础。数据集与工具均已公开，以支持 LLM 应用安全与开发方面的后续研究。
With the increasing adoption of large language models (LLMs), a growing number of applications leveraging these models have emerged. However, LLM-based applications face significant security challenges. The OWASP Top 10 for LLMs has identified critical vulnerabilities, including prompt injections, data leakage, and unauthorized code execution. While previous research has explored specific security issues, these studies often focus on a narrow range of problems and a small number of applications, leaving gaps in our understanding of security threats in real-world LLM-based applications. To facilitate research in this field, an automated tool was developed to systematically collect LLM applications and construct a dataset, particularly focusing on independent applications that leverage LLM APIs or frameworks. Our dataset includes 33,001 repositories, providing a broad and representative foundation for security analysis. Both our dataset and tool are publicly available to support further research and foster advancements in the security and development of LLM-based applications.
@inproceedings{paperbot8936,
title = {LLMAppHub: A Large Collection of LLM-based Applications for the Research Community},
author = {Zixuan Wu and Chen Zhi and Junxiao Han and Shuiguang Deng and Jianwei Yin},
booktitle = {FSE},
year = {2025},
doi = {10.1145/3696630.3731439}
}