paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Hailstorm

Hailstorm:解耦存储的分布式 LSM 数据库
ASPLOS Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems2020匹配度 0/10 · 该论文属于分布式数据库/存储系统方向,研究 LSM 数据库的负载均衡、存储计算分离与 compaction 卸载,与用户关注的程序分析、LLM/Agent、浮点程序分析、贝叶斯程序分析等方向均无交集。引用 50 inbox
Laurent Bindschaedler, Ashvin Goel, Willy Zwaenepoel
crossref: 2026-09-12

AI 速览

一句话:通过存储与计算分离、机架内存储池化和 compaction 远程卸载,解决分布式 LSM 数据库负载不均与后台任务干扰。

分布式 LSM 数据库常因实例间负载不均以及 flush、compaction 和数据迁移等后台任务的干扰,出现吞吐下降和延迟升高。Hailstorm 将数据库存储引擎部署在分布式文件系统上,把存储与处理解耦,实现存储池化与 compaction 卸载。它池化机架内的存储设备,使每个存储引擎都能利用整机架的聚合容量与带宽,从而在不重新分片的情况下应对负载不均;同时将 compaction 任务卸载到远程节点,分散其影响。评估表明,在倾斜负载的 MongoDB 部署中,Hailstorm 实现负载均衡,平均吞吐提升 60%,尾延迟降低至多 5 倍;范围查询吞吐提升至多 22 倍;OLTP 工作负载成本节省 47-56%。

问题
分布式 LSM 数据库因实例间负载不均和 flush、compaction、数据迁移等后台任务干扰,面临吞吐与延迟问题。
方法
Hailstorm 将存储引擎部署在分布式文件系统上,解耦存储与处理,池化机架内存储设备,并将 compaction 卸载到远程节点。
评估
在倾斜负载的 MongoDB 部署中评估,平均吞吐提升 60%,尾延迟降低至多 5 倍,范围查询吞吐提升至多 22 倍,OLTP 成本节省 47-56%。
原文摘要(Abstract)

Distributed LSM-based databases face throughput and latency issues due to load imbalance across instances and interference from background tasks such as flushing, compaction, and data migration. Hailstorm addresses these problems by deploying the database storage engines over a distributed filesystem that disaggregates storage from processing, enabling storage pooling and compaction offloading. Hailstorm pools storage devices within a rack, allowing each storage engine to fully utilize the aggregate rack storage capacity and bandwidth. Storage pooling successfully handles load imbalance without the need for resharding. Hailstorm offloads compaction tasks to remote nodes, distributing their impact, and improving overall system throughput and response time. We show that Hailstorm achieves load balance in many MongoDB deployments with skewed workloads, improving the average throughput by 60%, while decreasing tail latency by as much as 5X. In workloads with range queries, Hailstorm provides up to 22X throughput improvements. Hailstorm also enables cost savings of 47-56% in OLTP workloads.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot5477,
  title = {Hailstorm},
  author = {Laurent Bindschaedler and Ashvin Goel and Willy Zwaenepoel},
  booktitle = {ASPLOS},
  year = {2020},
  doi = {10.1145/3373376.3378504}
}