一句话：提出自底向上镶嵌式深度学习编译器 Mosaic 与 iTex 抽象，将张量计算异构映射到多种指令单元以挖掘指令级并行。
深度学习计算复杂度高，CPU/GPU虽提供标量、向量、张量等可并行单元，但现有自顶向下、基于tiling的编译器常生成同构映射，无法同时利用不同计算单元。本文提出Mosaic，一个自底向上、基于tessellation的深度学习编译器：用不同指令直接镶嵌张量计算任务，形成异构指令到任务映射以挖掘指令级并行（ILP）。其核心iTex抽象以形式化仿射函数建模指令操作与语义，并据此启发式生成镶嵌方案、调度指令执行，减少结构冒险并最大化ILP。多个硬件平台上，Mosaic相比厂商库平均加速1.08×–1.28×；在LLM真实算子上相比最佳基线加速1.34×，达GPU Tensor Core理论峰值吞吐106%。
Deep learning has achieved great success in numerous application areas at the cost of high computational complexity. To meet the ever-increasing computational demand, commodity hardware platforms (e.g., CPUs and GPUs) offer abundant computing resources including scalar, vector, and tensor units for deep learning that could execute in parallel. However, existing top-down tiling-based deep learning compilers often generate a homogeneous mapping from the given tensor computation task to hardware arithmetic instructions, failing to utilize different computing units simultaneously to achieve higher performance. In this paper, we propose Mosaic, a bottom-up tessellation-based deep learning compiler that directly tessellates the given tensor computation task with varying instructions, forming a heterogeneous instruction-to-task mapping to exploit instruction-level parallelism (ILP) across different computing units. The key that enables such tessellation is the iTex abstraction, which models the relationship between the instruction operations and its semantics with formalized affine functions. Based on the iTex, we propose a heuristic approach to efficiently generate various tessellation plans. Further, we propose the iTex scheduling technique to orchestrate the execution of instructions, reducing potential structural hazards and maximizing the exploitable ILP. Our extensive evaluation shows that Mosaic achieves an average speedup ranging from 1.08× to 1.28× across multiple hardware platforms compared to highly optimized vendor libraries. Mosaic also achieves an average speedup of 1.34× over the best existing baselines on real-world operators extracted from LLMs. More importantly, Mosaic reaches up to 106% of the GPU Tensor Core theoretical peak throughput, demonstrating its effective exploitation of ILP.
@inproceedings{XuWLXRB0HSH0DZJ25,
title = {Mosaic: Exploiting Instruction-Level Parallelism on Deep Learning Accelerators with
<i>iTex</i>
Tessellation},
author = {Jianxing Xu and Yuanbo Wen and Zikang Liu and Ruibai Xu and Tingfeng Ruan and Jun Bi and Rui Zhang and Di Huang and Xinkai Song and Yifan Hao and Xing Hu and Zidong Du and Chongqing Zhao and Jiang Jie and Qi Guo},
booktitle = {ASPLOS},
year = {2025},
doi = {10.1145/3676641.3716262}
}