一句话：提出知识引导的查询澄清方法，以 LLM 与知识图谱构建五步 AI 链，改进 API 推荐。
针对基于知识图谱的 API 查询澄清存在词汇外失败与模板僵化问题，本文提出一种由知识图谱引导大语言模型（LLM）的查询澄清方法。将 LLM 作为神经知识库以克服 OOV，并利用 KG 中结构化 API 知识与实体关系过滤噪声，同时将 KG 最优澄清路径迁移至 LLM 提升效率。该方法实现为五步 AI 链，每步调用一次 LLM。实验显示各单元评分接近满分 5；相比基线，MRR 最高提升 63.9%（KG 覆盖查询时）和 37.2%（未覆盖时）。消融表明 KG 引导与路径查找策略分别贡献 19.0% 和 22.2% 的 MAP 提升。该方法有效桥接 KG 与 LLM，互补优势。
API recommendation methods have evolved from literal and semantic keyword matching to query expansion and query clarification. The latest query clarification method is knowledge graph (KG)-based, but limitations include out-of-vocabulary (OOV) failures and rigid question templates. To address these limitations, we propose a novel knowledge-guided query clarification approach for API recommendation that leverages a large language model (LLM) guided by KG. We utilize the LLM as a neural knowledge base to overcome OOV failures, generating fluent and appropriate clarification questions and options. We also leverage the structured API knowledge and entity relationships stored in the KG to filter out noise, and transfer the optimal clarification path from KG to the LLM, increasing the efficiency of the clarification process. Our approach is designed as an AI chain that consists of five steps, each handled by a separate LLM call, to improve accuracy, efficiency, and fluency for query clarification in API recommendation. We verify the usefulness of each unit in our AI chain, which all received high scores close to a perfect 5. When compared to the baselines, our approach shows a significant improvement in MRR, with a maximum increase of 63.9% higher when the query statement is covered in KG and 37.2% when it is not. Ablation experiments reveal that the guidance of knowledge in the KG and the knowledge-guided pathfinding strategy are crucial for our approach's performance, resulting in a 19.0% and 22.2% increase in MAP, respectively. Our approach demonstrates a way to bridge the gap between KG and LLM, effectively compensating for the strengths and weaknesses of both.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{HuangWXWCXL23,
title = {Let's Chat to Find the APIs: Connecting Human, LLM and Knowledge Graph through AI Chain},
author = {Qing Huang and Zhenyu Wan and Zhenchang Xing and Changjing Wang and Jieshan Chen and Xiwei Xu and Qinghua Lu},
booktitle = {ASE},
year = {2023},
doi = {10.1109/ase56229.2023.00075}
}