一句话:提出 CloudRaid,通过挖掘历史日志翻转测试不足的消息顺序,自动检测云系统中的分布式并发缺陷。
云系统深受分布式并发缺陷困扰,这类缺陷极难检测,并常导致数据丢失与服务中断。本文提出 CloudRaid,一个通过日志挖掘自动检测云系统并发缺陷的工具。其核心观察是:大规模在线云应用每秒处理数百万用户请求,已充分测试的消息顺序不太可能暴露错误,因此应优先测试那些可行却测试不足的顺序。CloudRaid 从历史执行日志中挖掘消息顺序信息:若两条消息可能并行发生,但日志中总是 S 先于 P 到达,则尝试翻转二者顺序,实际执行 P ↣ S,从而触发潜在错误。该基于日志的方法无需插桩或离线重放,适合运行中的在线系统。
Cloud systems suffer from distributed concurrency bugs, which are notoriously difficult to detect and often lead to data loss and service outage. This paper presents CloudRaid, a new effective tool to battle distributed concurrency bugs. CloudRaid automatically detects concurrency bugs in cloud systems, by analyzing and testing those message orderings that are likely to expose errors. We observe that large-scale online cloud applications process millions of user requests per second, exercising many permutations of message orderings extensively. Those already sufficiently-tested message orderings are unlikely to expose errors. Hence, CloudRaid mines logs from previous executions to uncover those message orderings which are feasible, but not sufficiently tested. Specifically, CloudRaid tries to flip the order of a pair of messages if they may happen in parallel, but S always arrives before P from existing logs, i.e., excercising the order P ↣ S. The log-based approach makes it suitable to live systems.
DOI 原文 ·
@inproceedings{paperbot4584,
title = {CloudRaid: hunting concurrency bugs in the cloud via log-mining},
author = {Jie Lu and Feng Li and Lian Li and Xiaobing Feng},
booktitle = {FSE},
year = {2018},
doi = {10.1145/3236024.3236071}
}