一句话：提出缺陷注入系统 Apocalypse，用符号执行与约束合成生成公平、深层、独立、可复现且罕见的缺陷语料。
评估缺陷检测工具长期缺乏可靠的真实缺陷语料；已有 LAVA、EvilCoder 等自动缺陷注入系统生成的缺陷与自然缺陷存在多方面差异。本文提出自动缺陷注入系统 Apocalypse，利用符号执行、基于约束的程序合成与模型计数等形式化技术，自动向大型代码库注入公平（可被现有工具发现）、深层（触发需满足长依赖链）、互不相关、可复现且罕见（仅少量输入可触发）的缺陷。作者将缺陷注入 30 个 Coreutils 程序以及 TCAS 测试套件，并从多个指标评估其真实性。结果表明 Apocalypse 合成的缺陷高度真实，不偏向某一类缺陷发现策略，且比手工注入的缺陷更难发现：最先进的符号执行工具需多约 240 倍的测试才能发现。
In spite of decades of research in bug detection tools, there is a surprising dearth of ground-truth corpora that can be used to evaluate the efficacy of such tools. Recently, systems such as LAVA and EvilCoder have been proposed to automatically inject bugs into software to quickly generate large bug corpora, but the bugs created so far differ from naturally occurring bugs in a number of ways. In this work, we propose a new automated bug injection system, Apocalypse, that uses formal techniques—symbolic execution, constraint-based program synthesis and model counting—to automatically inject fair (can potentially be discovered by current bug-detection tools), deep (requiring a long sequence of dependencies to be satisfied to fire), uncorrelated (each bug behaving independent of others), reproducible (a trigger input being available) and rare (can be triggered by only a few program inputs) bugs in large software code bases. In our evaluation, we inject bugs into thirty Coreutils programs as well as the TCAS test suite. We find that bugs synthesized by Apocalypse are highly realistic under a variety of metrics, that they do not favor a particular bug-finding strategy (unlike bugs produced by LAVA), and that they are more difficult to find than manually injected bugs, requiring up around 240× more tests to discover with a state-of-the-art symbolic execution tool.
DOI 原文 ·
@inproceedings{paperbot4619,
title = {Bug synthesis: challenging bug-finding tools with deep faults},
author = {Subhajit Roy and Awanish Pandey and Brendan Dolan-Gavitt and Yu Hu},
booktitle = {FSE},
year = {2018},
doi = {10.1145/3236024.3236084}
}