一句话：提出 GMTA，用图表示与混合存储实时处理微服务 trace，支持架构理解与问题诊断，并在 eBay 部署验证。
微服务系统高度动态且复杂，运维与开发人员依赖 trace 分析来理解架构、诊断服务失败与质量退化等问题，但运行时产生的海量 trace 使实时捕获所需信息十分困难。本文提出基于图的微服务 trace 分析方案 GMTA：以图结构表示 trace，将 trace 抽象为不同路径并进一步分组为业务流；结合图数据库与实时分析数据库，设计高效存储与访问机制，以支撑多种分析应用。基于 GMTA 构建了架构理解与问题诊断应用，支持服务依赖可视化、架构决策、服务行为变化分析、性能问题检测与根因定位。GMTA 已在 eBay 实现并部署，基于 eBay trace 数据的实验验证了其有效性与效率，监控团队和 SRE 团队的案例研究进一步确认了其在大规模工业微服务系统中的价值。
Microservice systems are highly dynamic and complex. For such systems, operation engineers and developers highly rely on trace analysis to understand architectures and diagnose various problems such as service failures and quality degradation. However, the huge number of traces produced at runtime makes it challenging to capture the required information in real-time. To address the faced challenges, in this paper, we propose a graph-based microservice trace analysis approach GMTA for understanding architecture and diagnosing various problems. Built on a graph-based representation, GMTA includes efficient processing of traces produced on the fly. It abstracts traces into different paths and further groups them into business flows. To support various analytical applications, GMTA includes an efficient storage and access mechanism by combining a graph database and a real-time analytics database and using a carefully designed storage structure. Based on GMTA, we construct analytical applications for architecture understanding and problem diagnosis, these applications support various needs such as visualizing service dependencies, making architectural decisions, analyzing the changes of services behaviors, detecting performance issues, and locating root causes. GMTA has been implemented and deployed in eBay. An experimental study based on trace data produced by eBay demonstrates GMTA's effectiveness and efficiency for architecture understanding and problem diagnosis. Case studies conducted in eBay's monitoring team and Site Reliability Engineering (SRE) team further confirm GMTA's substantial benefits in industrial-scale microservice systems.
DOI 原文 ·
@inproceedings{paperbot5842,
title = {Graph-based trace analysis for microservice architecture understanding and problem diagnosis},
author = {Xiaofeng Guo and Xin Peng and Hanzhang Wang and Wanxue Li and Huai Jiang and Dan Ding and Tao Xie and Liangfei Su},
booktitle = {FSE},
year = {2020},
doi = {10.1145/3368089.3417066}
}