一句话：结合机器学习与程序合成，将非结构化的排障指南（TSG）自动转换为可执行工作流。
在大型云服务运维中，值班工程师（OCE）依赖排障指南（TSG）快速处理事故，但 TSG 彼此孤立、非结构化且常不完整，需人工理解并逐步执行，导致值班疲劳、效率下降和人为错误。本文先对 4K+ 条与事故关联的 TSG 做大规模实证研究，发现 TSG 使用广泛且能显著缩短缓解时间；随后分析 400+ 名 OCE 的反馈，提出 TSG 质量问题的分类法。为弥补这些缺陷，作者提出 AutoTSG——一个将 TSG 自动化为可执行工作流的框架，融合机器学习与程序合成：先用分类模型识别 TSG 中可执行的语句，再将其解析合成为可执行工作流。在 50 条 TSG 上的评估显示，语句识别准确率为 0.89，解析精确率 0.94、召回率 0.91；对 10 名微软工程师的调研进一步表明 TSG 自动化的重要性与 AutoTSG 的实用性。
Incident management is a key aspect of operating large-scale cloud services. To aid with faster and efficient resolution of incidents, engineering teams document frequent troubleshooting steps in the form of Troubleshooting Guides (TSGs), to be used by on-call engineers (OCEs). However, TSGs are siloed, unstructured, and often incomplete, requiring developers to manually understand and execute necessary steps. This results in a plethora of issues such as on-call fatigue, reduced productivity, and human errors. In this work, we conduct a large-scale empirical study of over 4K+ TSGs mapped to incidents and find that TSGs are widely used and help significantly reduce mitigation efforts. We then analyze feedback on TSGs provided by 400+ OCEs and propose a taxonomy of issues that highlights significant gaps in TSG quality. To alleviate these gaps, we investigate the automation of TSGs and propose AutoTSG -- a novel framework for automation of TSGs to executable workflows by combining machine learning and program synthesis. Our evaluation of AutoTSG on 50 TSGs shows the effectiveness in both identifying TSG statements (accuracy 0.89) and parsing them for execution (precision 0.94 and recall 0.91). Lastly, we survey ten Microsoft engineers and show the importance of TSG automation and the usefulness of AutoTSG.
DOI 原文 ·
@inproceedings{paperbot7038,
title = {AutoTSG: learning and synthesis for incident troubleshooting},
author = {Manish Shetty and Chetan Bansal and Sai Pramod Upadhyayula and Arjun Radhakrishna and Anurag Gupta},
booktitle = {FSE},
year = {2022},
doi = {10.1145/3540250.3558958}
}