一句话：提出基于 LLM 的 AndroB2O，为 Android 缺陷报告自动生成断言式失败测试预言。
移动应用缺陷报告管理耗时费力，其中为 GUI 功能失败创建测试预言仍缺乏自动化支持。论文先实证研究 LLM 多模态（文本与 GUI 代码）推理能否生成非崩溃功能失败的断言预言，据此提出 AndroB2O：输入缺陷报告与相关 GUI 屏幕，先定位失败相关 GUI 元素，再依据元素属性生成旨在确认失败不存在的断言。作者构建首个包含 GUI 交互与揭示报告失败断言的 Android 缺陷报告数据集，在 152 个失败上评估，AndroB2O 在 61.2% 案例中生成能成功识别失败的 FBO；与失败复现工具 ReBL 集成后，自动生成含预言的测试用例取得有前景结果。
In the context of mobile apps, bug report management tasks have been shown to be among the most time-consuming and intellectually intensive software maintenance activities. As such, researchers have developed tools to automate the reproduction, validation, and localization of reported bugs. However, one complex, time-consuming, and important task that lacks automated support is the creation of test oracles for reported functional failures that manifest through the GUI. This is challenging task – requiring nuanced, multi-modal reasoning about bug descriptions, affected GUI components, and the characteristics of the related erroneous program state(s).To explore the feasibility of automating this task, we conduct a empirical investigation into how the multi-modal (i.e., text and GUI-related code) reasoning capabilities of Large Language Models (LLMs) can be used to automatically generate assertion-based test oracles for non-crashing, functional failures described in Android app bug reports. Building upon the findings of this study, we construct and evaluate AndroB2O, an automated, LLM-based approach that, given a bug report and the GUI screen associated with the reported failure as inputs, generates failure-based oracles (FBOs) in the form of test assertions. The approach first identifies the GUI elements related to the failure and then defines assertions that aim to confirm the absence of the failure based on the elements’ properties. To evaluate AndroB2O, we create the first dataset of Android bug reports containing test cases with GUI interactions and test oracles that reveal reported failures. The results of our evaluation on 152 failures show that AndroB2O is able to generate FBOs that successfully identify the failure (and hence can confirm its absence) in 61.2% of the cases. We integrated AndroB2O with ReBL, a failure reproduction tool, to evaluate its effectiveness in automated generation of test cases complete with oracles for reported failures, and obtained promising results.
DOI 原文 ·
@inproceedings{paperbot9513,
title = {Generating Failure-Based Oracles to Support Testing of Reported Bugs in Android Apps},
author = {Jack Johnson and Junayed Mahmud and Oscar Chaparro and Kevin Moran and Mattia Fazzini},
booktitle = {ASE},
year = {2025},
doi = {10.1109/ase63991.2025.00182}
}