一句话：提出 PtxDec，用编译器数据增强与循环重卷中间表示，显著提升 LLM 将优化 PTX 反编译为可读 CUDA 的准确率。
面向 GPU 上高性能张量程序（尤其是大语言模型负载），分析经高度优化的底层 PTX 代码对性能调优与理解至关重要，但门槛很高。LLM 有潜力做 PTX 到 CUDA 的反编译，却受限于对齐训练数据稀缺以及优化后代码（如循环展开）的复杂性。本文提出 PtxDec 原型：一方面构建基于编译器的数据增强框架并配合严格后处理，生成 40 万对高质量对齐的 CUDA-PTX kernel 数据用于训练；另一方面引入 Rolled-PTX 这一中间表示，在预处理阶段用启发式循环重卷折叠展开模式，简化输入结构并使之更贴近高层循环结构。评估显示，相较基线方法功能准确率提升 2.3–3.1 倍，生成代码可读性及与原优化 kernel 的调度一致性也明显改善，消融实验验证了各组件的贡献。据作者所知，这是首个面向 PTX-to-CUDA 反编译、并系统性增强 LLM 以应对该领域关键挑战的工作。
The growing demand for high-performance tensor programs on GPUs, especially for large language models (LLMs), necessitates advanced compilation and optimization techniques. However, the critical task of analyzing optimized, low-level PTX code for performance tuning or understanding poses significant challenges. While LLMs hold promise for PTX-to-CUDA de-compilation to improve code intelligibility, their effectiveness is severely limited by the scarcity of aligned training data and the inherent complexity of highly optimized, unrolled PTX code.In this work, we explore methodologies to significantly enhance LLM capabilities for accurate and readable PTX-to-CUDA decompilation and present PtxDec, a decompilation prototype implementing our approach. To overcome the critical barrier of data scarcity, we develop a compiler-based data augmentation framework coupled with rigorous post-processing, enabling the creation of a large-scale, high-quality dataset of 400K aligned CUDA-PTX kernel pairs for effective LLM training. Furthermore, to empower LLMs to handle the complexity of optimized PTX, we introduce Rolled-PTX—an intermediate representation generated through heuristic loop rerolling during preprocessing. Rolled-PTX condenses unrolled patterns, drastically simplifying the input structure presented to the LLM and aligning it better with higher-level loop constructs.Comprehensive evaluation demonstrates that PtxDec achieves substantial performance gains: our approach yields a 2.3×–3.1× improvement in functional accuracy over baseline methods, alongside significant enhancements in generated code readability and scheduling consistency with the original optimized kernels. Ablation studies further validate the contribution of each proposed component to the overall performance.To the best of our knowledge, this is the first work tackling PTX-to-CUDA decompilation, specifically focusing on and demonstrating effective strategies for augmenting LLMs to overcome the key challenges in this domain.
DOI 原文 ·
@inproceedings{paperbot9273,
title = {Enhancing LLM to Decompile Optimized PTX to Readable CUDA for Tensor Programs},
author = {Xinyu Sun and Fugen Tang and Yu Zhang and Han Shen and Chengru Song and Di Zhang},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00185}
}