一句话：利用编译器优化与反馈引导学习，从源码自动生成大量多样二进制，用于测试二进制分析工具。
二进制分析是安全与软件工程的重要能力，但测试分析工具需要大量多样且带源码信息的二进制数据集。本文提出 Cornucopia：一个架构无关的自动化框架，利用编译器优化与反馈引导学习，从源码生成大量二进制变体。评估中，它跨 x86、x64、ARM、MIPS 生成 309K 个二进制，平均每程序 403 个，优于 BinTuner，并暴露约 300 次 LLVM 优化调度器导致的编译崩溃。用这些二进制测试 angr、Ghidra、ida、radare，发现 angr 263 次崩溃、ida 一处内存破坏及多种语义 bug（差分测试）；Asm2Vec、SAFE、Debin 等机器学习工具表现下降，如 Debin F1 从 63.1% 降至 12.9%。
Binary analysis is an important capability required for many security and software engineering applications. Consequently, there are many binary analysis techniques and tools with varied capabilities. However, testing these tools requires a large, varied binary dataset with corresponding source-level information. In this paper, we present Cornucopia, an architecture agnostic automated framework that can generate a plethora of binaries from corresponding program source by exploiting compiler optimizations and feedback-guided learning. Our evaluation shows that Cornucopia was able to generate 309K binaries across four architectures (x86, x64, ARM, MIPS) with an average of 403 binaries for each program and outperforms BinTuner [53], a similar technique. Our experiments revealed issues with the LLVM optimization scheduler resulting in compiler crashes (∼ 300). Our evaluation of four popular binary analysis tools angr, Ghidra, ida, and radare, using Cornucopia generated binaries, revealed various issues with these tools. Specifically, we found 263 crashes in angr and one memory corruption issue in ida. Our differential testing on the analysis results revealed various semantic bugs in these tools. We also tested machine learning tools, Asm2Vec, SAFE, and Debin, that claim to capture binary semantics and show that they perform poorly (e.g., Debin F1 score dropped to 12.9% from reported 63.1%) on Cornucopia generated binaries. In summary, our exhaustive evaluation shows that Cornucopia is an effective mechanism to generate binaries for testing binary analysis techniques effectively.
@inproceedings{paperbot7225,
title = {Cornucopia : A Framework for Feedback Guided Generation of Binaries},
author = {Vidush Singhal and Akul Abhilash Pillai and Charitha Saumya and Milind Kulkarni and Aravind Machiry},
booktitle = {ASE},
year = {2022},
doi = {10.1145/3551349.3561152}
}