一句话：提出 CLAP，用自然语言监督对齐汇编代码与语义解释，显著提升二进制代码表示的少样本与零样本迁移能力。
二进制代码表示学习在二进制分析中已取得不错效果，但现有方法迁移性差，尤其在少样本和零样本场景。本文提出 CLAP（Contrastive Language-Assembly Pre-training），用自然语言监督学习汇编代码表示：将二进制代码与其自然语言语义解释对齐，使模型生成更好的二进制代码嵌入，从而提升迁移能力。为支持对齐训练，作者设计了高效数据集引擎，自动生成大规模、多样化的代码—解释配对，构建 1.95 亿对数据并训练 CLAP 原型。在多种二进制分析下游任务上，CLAP 表现优异；在无任务特定训练时，其性能常可与全监督基线竞争，展现优秀迁移性。
Binary code representation learning has shown significant performance in binary analysis tasks. But existing solutions often have poor transferability, particularly in few-shot and zero-shot scenarios where few or no training samples are available for the tasks. To address this problem, we present CLAP (Contrastive Language-Assembly Pre-training), which employs natural language supervision to learn better representations of binary code (i.e., assembly code) and get better transferability. At the core, our approach boosts superior transfer learning capabilities by effectively aligning binary code with their semantics explanations (in natural language), resulting a model able to generate better embeddings for binary code. To enable this alignment training, we then propose an efficient dataset engine that could automatically generate a large and diverse dataset comprising of binary code and corresponding natural language explanations. We have generated 195 million pairs of binary code and explanations and trained a prototype of CLAP. The evaluations of CLAP across various downstream tasks in binary analysis all demonstrate exceptional performance. Notably, without any task-specific training, CLAP is often competitive with a fully supervised baseline, showing excellent transferability.
@inproceedings{paperbot8511,
title = {CLAP: Learning Transferable Binary Code Representations with Natural Language Supervision},
author = {Hao Wang and Zeyu Gao and Chao Zhang and Zihan Sha and Mingyang Sun and Yuchen Zhou and Wenyu Zhu and Wenju Sun and Han Qiu and Xi Xiao},
booktitle = {ISSTA},
year = {2024},
doi = {10.1145/3650212.3652145}
}