一句话：提出基于 eBPF 的非侵入式、语言无关日志削减框架，在线识别并降低日志热点开销。
现代系统产生海量日志以检测和诊断故障，带来高昂存储成本与运行时开销。作者调查真实生产日志后发现，多数日志开销由少量日志模板（日志热点）引起，并以微信工业系统为对象进行系统研究。为此，论文提出 LogReducer：一个基于 eBPF 的非侵入式、语言无关日志削减框架，包含在线与离线流程，可即时识别并减少日志热点。离线流程在微信服务两个月后，日志存储开销从每天 19.7 PB 降至 12.0 PB，下降约 39.08%；测试环境表明在线流程能在保留日志有效性的同时控制热点日志开销，并将日志热点处理时间从生产环境平均 9 天缩短至测试中的 10 分钟。
Modern systems generate a massive amount of logs to detect and diagnose system faults, which incurs expensive storage costs and runtime overhead. After investigating real-world production logs, we observe that most of the logging overhead is due to a small number of log templates, referred to as log hotspots. Therefore, we conduct a systematical study about log hotspots in an industrial system WeChat, which motivates us to identify log hotspots and reduce them on the fly. In this paper, we propose LogReducer, a non-intrusive and language-independent log reduction framework based on eBPF (Extended Berkeley Packet Filter), consisting of both online and offline processes. After two months of serving the offline process of LogReducer in WeChat, the log storage overhead has dropped from 19.7 PB per day to 12.0 PB (i.e., about a 39.08% decrease). Practical implementation and experimental evaluations in the test environment demonstrate that the online process of LogReducer can control the logging overhead of hotspots while preserving logging effectiveness. Moreover, the log hotspot handling time can be reduced from an average of 9 days in production to 10 minutes in the test with the help of LogReducer,
@inproceedings{YuCLWZDZ23,
title = {LogReducer: Identify and Reduce Log Hotspots in Kernel on the Fly},
author = {Guangba Yu and Pengfei Chen and Pairui Li and Tianjun Weng and Haibing Zheng and Yuetang Deng and Zibin Zheng},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse48619.2023.00151}
}