一句话：提出 Janus，用视觉 Transformer 场景学习与视频对齐检测基于视频的重复缺陷报告。
基于视频的缺陷报告常用于记录 GUI 程序缺陷，但自动管理这类报告需要识别理解其中细微的视觉模式，重复检测尤为困难。本文提出 Janus，将视觉 Transformer 的场景学习能力用于捕捉 App UI 屏幕上细微的视觉与文本模式，以区分相似界面；同时引入视频对齐技术，自适应地为视频帧加权以适配典型缺陷表现模式。在包含 270 份视频缺陷报告、源自 90 个 Android 应用缺陷的 7,290 个重复检测任务上，最佳配置取得 89.8%/84.7% 的 mRR/mAP，且在大多数任务上以约 9% 的显著幅度优于先前工作。定性分析展示了场景学习能力对性能的助益。
Video-based bug reports are increasingly being used to document bugs for programs centered around a graphical user interface (GUI). However, developing automated techniques to manage video-based reports is challenging as it requires identifying and understanding often nuanced visual patterns that capture key information about a reported bug. In this paper, we aim to overcome these challenges by advancing the bug report management task of duplicate detection for video-based reports. To this end, we introduce a new approach, called Janus, that adapts the scene-learning capabilities of vision transformers to capture subtle visual and textual patterns that manifest on app UI screens --- which is key to differentiating between similar screens for accurate duplicate report detection. Janus also makes use of a video alignment technique capable of adaptive weighting of video frames to account for typical bug manifestation patterns. In a comprehensive evaluation on a benchmark containing 7,290 duplicate detection tasks derived from 270 video-based bug reports from 90 Android app bugs, the best configuration of our approach achieves an overall mRR/mAP of 89.8%/84.7%, and for the large majority of duplicate detection tasks, outperforms prior work by ≈9% to a statistically significant degree. Finally, we qualitatively illustrate how the scene-learning capabilities provided by Janus benefits its performance.
@inproceedings{paperbot11394,
title = {Semantic GUI Scene Learning and Video Alignment for Detecting Duplicate Video-based Bug Reports},
author = {Yanfu Yan and Nathan Cooper and Oscar Chaparro and Kevin Moran and Denys Poshyvanyk},
booktitle = {ICSE},
year = {2024},
doi = {10.1145/3597503.3639163}
}