一句话：提出从缺陷报告抽取用户任务与失败的开放信息抽取方法，构建系统知识图谱并生成肥皂剧测试场景。
探索式测试依赖测试者的知识与创造力，但自动生成探索式测试场景（肥皂剧测试）缺乏有效工具，现有基于规约、程序差异和模糊测试的技术并不适用。本文提出利用缺陷报告中的场景与预期知识生成肥皂剧测试场景，开发开放信息抽取方法，从缺陷报告的复现步骤、期望结果与观察结果中抽取用户任务与失败，构建系统知识图谱。作者基于 Firefox 的 25,939 个缺陷构建概念验证知识图谱，评估显示质量较高。基于该图谱组合相关缺陷场景生成肥皂剧测试场景，并实现 Web 工具支持探索式测试。用户研究中，5 名用户 2 小时内从 5 个种子缺陷发现 18 个，对照组仅 5 个。
Exploratory testing is an effective testing approach which leverages the tester’s knowledge and creativity to design test cases to provoke and recognize failures at the system level from the end user’s perspective. Although some principles and guidelines have been proposed to guide exploratory testing, there are no effective tools for automatic generation of exploratory test scenarios (a.k.a soap opera tests). Existing test generation techniques rely on specifications, program differences and fuzzing, which are not suitable for exploratory test generation. In this paper, we propose to leverage the scenario and oracle knowledge in bug reports to generate soap opera test scenarios. We develop open information extraction methods to construct a system knowledge graph (KG) of user tasks and failures from the steps to reproduce, expected results and observed results in bug reports. We construct a proof-of-concept KG from 25,939 bugs of the Firefox browser. Our evaluation shows the constructed KG is of high quality. Based on the KG, we create soap opera test scenarios by combining the scenarios of relevant bugs, and develop a web tool to present the created test scenarios and support exploratory testing. In our user study, 5 users find 18 bugs from 5 seed bugs in 2 hours using our tool, while the control group finds only 5 bugs based on the recommended similar bugs.
@inproceedings{paperbot7269,
title = {Constructing a System Knowledge Graph of User Tasks and Failures from Bug Reports to Support Soap Opera Testing},
author = {Yanqi Su and Zheming Han and Zhenchang Xing and Xin Xia and Xiwei Xu and Liming Zhu and Qinghua Lu},
booktitle = {ASE},
year = {2022},
doi = {10.1145/3551349.3556967}
}