一句话:提出 AutoDoc,利用微调稠密检索与 GPT-4o 从 Stack Overflow 自动生成准确、低冗余的 API 文档。
官方 API 文档常存在过时和不完整的问题。该文提出 AutoDoc,从 Stack Overflow 讨论中抽取 API 知识来自动生成文档:先用微调稠密检索模型识别帖子中的七类 API 知识,再用 GPT-4o 将其摘要为简洁文本,并设计两个专门组件缓解 LLM 幻觉与内容冗余。作者在 48 个不同流行度的 API 上与五个基线对比,发现 AutoDoc 生成的文档准确率最高提升 77.7%,重复内容减少 9.5%,且包含 34.4% 官方文档未覆盖的知识。研究还分析了不同 LLM 的敏感性,并开展用户研究;参与者认为 AutoDoc 文档更全面、简洁且有用。
API documentation is crucial for developers to learn and use APIs. However, it is known that many official API documents are obsolete and incomplete. To address this challenge, we propose a new approach called AutoDoc that generates API documents with API knowledge extracted from online discussions on Stack Overflow (SO). AutoDoc leverages a fine-tuned dense retrieval model to identify seven types of API knowledge from SO posts. Then, it uses GPT-4o to summarize the API knowledge in these posts into concise text. Meanwhile, we designed two specific components to handle LLM hallucination and redundancy in generated content. We evaluated AutoDoc against five comparison baselines on 48 APIs of different popularity levels. Our results indicate that the API documents generated by AutoDoc are up to 77.7% more accurate, 9.5% less duplicated, and contain 34.4% knowledge uncovered by the official documents. We also measured the sensitivity of AutoDoc to the choice of different LLMs. We found that while larger LLMs produce higher-quality API documents, AutoDoc enables smaller open-source models (e.g., Mistral-7B-v0.3) to achieve comparable results. Finally, we conducted a user study to evaluate the usefulness of the API documents generated by AutoDoc. All participants found API documents generated by AutoDoc to be more comprehensive, concise, and helpful than the comparison baselines. This highlights the feasibility of utilizing LLMs for API documentation with careful design to counter LLM hallucination and information redundancy.