一句话:提出输入正确性与有趣性度量,并探索生成既正确又有趣的输入以测试输入校验后的程序逻辑。
当前测试生成器产出的输入受限,导致程序测试不均衡,尤其面对期望高度结构化输入的程序时,大量代码未获充分测试。本文提出两种度量以评估不同测试生成方法并定位未被充分测试的程序部分:输入正确性衡量输入满足期望结构的程度,可对照实现或规约判断;输入有趣性衡量输入到等价类边界的距离。基于对 clang 的初步评估,作者给出输入正确性分布,表明以生成结构合法输入为目标的方法难以测试输入校验步骤之后的程序逻辑。最后提出三种方法,试图生成既正确又有趣的输入,以弥补这一缺口。
Current test generators are limited in the inputs they produce. This leads to an imbalance in the program parts tested, with significant program parts receiving insufficient attention, particularly with programs expecting inputs that are highly structured. In this work, I propose two measurements to evaluate different approaches and find which parts of programs are under-tested. Input correctness provides a level to which an input satisfies the expected structure, measured either against an implementation or specification. Input interestingness checks the distance to equivalence class boundaries. I present the distribution of input correctness from an initial evaluation on a clang, indicating that approaches designed to provide structurally valid inputs are unable to test program logic behind the input validation steps. I finally propose three approaches to closing this gap by producing inputs that are both correct and interesting.