一句话:将自动化测试生成建模为不确定下的序贯决策问题,用 MCTS 结合定向模糊测试,在 Test-Comp 2020 覆盖基准上验证有效性。
具体执行与模糊测试是两类互补的覆盖率测试技术,如何兼取二者之长仍是开放问题。本文提出并评估 Legion:它改造 AI 领域的蒙特卡洛树搜索(MCTS),把自动化测试生成建模为不确定条件下的序贯决策问题,用最佳优先搜索根据历史迭代的奖励,学习每轮最值得探索的程序状态。为探索 MCTS 选中的状态,Legion 引入称为近似路径保持模糊测试(APPFuzzing)的定向模糊测试,作为蒙特卡洛模拟手段,由扩展已有约束采样工作实现。作者在 Test-Comp 2020 覆盖率类别的 2531 个基准上与竞争对手比较,并分析超参数敏感性,显示 Legion 在多种输入程序上有效。
Concolic execution and fuzzing are two complementary coverage-based testing techniques. How to achieve the best of both remains an open challenge. To address this research problem, we propose and evaluate Legion. Legion re-engineers the Monte Carlo tree search (MCTS) framework from the AI literature to treat automated test generation as a problem of sequential decision-making under uncertainty. Its best-first search strategy provides a principled way to learn the most promising program states to investigate at each search iteration, based on observed rewards from previous iterations. Legion incorporates a form of directed fuzzing that we call approximate path-preserving fuzzing (APPFuzzing) to investigate program states selected by MCTS. APPFuzzing serves as the Monte Carlo simulation technique and is implemented by extending prior work on constrained sampling. We evaluate Legion against competitors on 2531 benchmarks from the coverage category of Test-Comp 2020, as well as measuring its sensitivity to hyperparameters, demonstrating its effectiveness on a wide variety of input programs.