一句话：将提示优化用于云事故根因分析，显著提升LLM与微调SLM推荐准确率。
大规模云系统事故的根因分析（RCA）知识密集、依赖人工，现有基于LLM的RCA多采用默认微调或RAG配合静态人工提示，推荐效果受限。本文利用PromptWizard自动优化提示指令，并结合语义相似的历史案例查询底层LLM；同时基于Microsoft超18万条历史事故数据微调低成本小语言模型（SLM），验证提示优化在领域适应模型上的作用。在3K测试事故上，提示优化使RCA推荐准确率较RAG-LLM和微调SLM分别提升21%与13%；事故负责人参与的人工评估也验证其有效性，且未增加计算开销。
Root cause analysis (RCA) for incidents in large-scale cloud systems is a complex, knowledge-intensive task that often requires significant manual effort from on-call engineers (OCEs). Improving RCA is vital for accelerating the incident resolution process and reducing service downtime and manual efforts. Recent advancements in Large-Language Models (LLMs) have proven to be effective in solving different stages of the incident management lifecycle including RCA. However, existing LLM-based RCA recommendations typically leverage default finetuning or retrieval augmented generation (RAG) methods with static, manually designed prompts, which lead to sub-optimal recommendations. In this work, we leverage 'PromptWizard', a state-of-the-art prompt optimization technique, to automatically identify the best optimized prompt instruction that is combined with semantically similar historical examples for querying underlying LLMs during inference. Moreover, by utilizing more than 180K historical incident data from Microsoft, we developed cost-effective finetuned small language models (SLMs) for RCA recommendation generation and demonstrate the power of prompt optimization on such domain-adapted models. Our extensive experimental results show that prompt optimization can improve the accuracy of RCA recommendations by 21% and 13% on 3K test incidents over RAG-based LLMs and finetuned SLMs, respectively. Lastly, our human evaluation with incident owners have demonstrated the efficacy of prompt optimization on RCA recommendation tasks. These findings underscore the advantages of incorporating prompt optimization into AI for Operations (AIOps) systems, delivering substantial gains without increasing computational overhead.
@inproceedings{abs-2504-11505,
title = {eARCO: Efficient Automated Root Cause Analysis with Prompt Optimization},
author = {Drishti Goel and Raghav Magazine and Supriyo Ghosh and Akshay Nambi and Prathamesh Deshpande and Xuchao Zhang and Chetan Bansal and Saravan Rajmohan},
booktitle = {ASE},
year = {2026},
doi = {10.1145/3832783.3834477}
}