paperbot · PL 论文追踪

全部论文 待读 收藏

P2 Khuzdul: Efficient and Scalable Distributed Graph Pattern Mining Engine

Khuzdul:高效可扩展的分布式图模式挖掘引擎
ASPLOS Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 22023匹配度 1/10 · 该论文研究分布式图模式挖掘引擎,属于并行与分布式系统/图计算方向,与用户关注的静态分析、LLM/Agent、浮点程序分析、贝叶斯程序分析等核心方向没有直接关联。引用 16 inbox
Jingji Chen, Xuehai Qian
crossref: 2026-09-12

AI 速览

一句话:提出 Khuzdul 分布式执行引擎,通过可扩展嵌入抽象与 BFS-DFS 混合探索,实现高效可扩展的分布式图模式挖掘。

现有分布式图模式挖掘(GPM)系统在划分图上性能与可扩展性难以兼顾,复制图方案则受限于内存与通信开销。本文提出 Khuzdul,一种具有明确抽象的分布式执行引擎,可与现有单机 GPM 系统集成。其核心是可扩展嵌入抽象,能表达模式枚举算法、支持细粒度任务调度,并实现低成本 GPM 专用数据复用以降低通信开销。BFS-DFS 混合探索生成充足并发任务,实现通信-计算重叠且内存消耗有界。作者将 Automine 和 GraphPi 移植到 Khuzdul 上构建两个分布式系统。评估表明,基于 Khuzdul 的系统显著优于最先进的划分图分布式 GPM 系统,最高加速 75.5 倍(平均 19.0 倍),与复制图最快系统性能相当或更优,并可在 commodity cluster 上扩展到超过一千亿条边的图。

问题
现有分布式图模式挖掘系统在划分图上的性能与可扩展性难以兼顾,复制图方案则受限于内存与通信开销。
方法
提出 Khuzdul 分布式执行引擎,核心是可扩展嵌入抽象,支持模式枚举算法表达、细粒度任务调度与低成本 GPM 专用数据复用;采用 BFS-DFS 混合探索生成并发任务,重叠通信与计算并控制内存。
评估
通过将 Automine 和 GraphPi 移植到 Khuzdul 实现两个分布式 GPM 系统,与最先进系统对比,在划分图上最高加速 75.5 倍(平均 19.0 倍),与复制图最快系统性能相当或更优,并可扩展到超千亿条边的图。
原文摘要(Abstract)

This paper proposes Khuzdul, a distributed execution engine with a well-defined abstraction that can be integrated with existing single-machine graph pattern mining (GPM) systems to provide efficiency and scalability at the same time. The key novelty is the extendable embedding abstraction which can express pattern enumeration algorithms, allow fine-grained task scheduling, and enable low-cost GPM-specific data reuse to reduce communication cost. The effective BFS-DFS hybrid exploration generates sufficient concurrent tasks for communication-computation overlapping with bounded memory consumption. Two scalable distributed GPM systems are implemented by porting Automine and GraphPi on Khuzdul. Our evaluation shows that Khuzdul based systems significantly outperform state-of-the-art distributed GPM systems with partitioned graphs by up to 75.5× (on average 19.0×), achieve similar or even better performance compared with the fastest distributed GPM systems with replicated graph, and scale to massive graphs with more than one hundred billion edges with a commodity cluster.

链接与引用

DOI 原文 · PDF(开放获取) ·

BibTeX
@inproceedings{paperbot10030,
  title = {Khuzdul: Efficient and Scalable Distributed Graph Pattern Mining Engine},
  author = {Jingji Chen and Xuehai Qian},
  booktitle = {ASPLOS},
  year = {2023},
  doi = {10.1145/3575693.3575743}
}