一句话：提出基于图像处理与卷积网络的轻量级方法 CAPdroid，自动为移动应用缺陷录屏生成用户操作字幕。
移动应用缺陷报告常附带屏幕录制，其中包含丰富信息，但开发者逐帧观看并理解用户操作既耗时又枯燥。受电影字幕启发，论文提出轻量级方法 CAPdroid，自动为缺陷录屏生成字幕。CAPdroid 是纯图像、非侵入式方法，结合图像处理与卷积深度模型，对录屏进行分段、推断用户动作属性并生成字幕描述。自动化实验表明，CAPdroid 从录屏推断用户操作表现良好；用户研究也证实，其生成的步骤描述有助于开发者进行缺陷复现。
Screen recordings of mobile applications are easy to capture and include a wealth of information, making them a popular mechanism for users to inform developers of the problems encountered in the bug reports. However, watching the bug recordings and efficiently understanding the semantics of user actions can be time-consuming and tedious for developers. Inspired by the conception of the video subtitle in movie industry, we present a lightweight approach CAPdroid to caption bug recordings automatically. CAPdroid is a purely image-based and non-intrusive approach by using image processing and convolutional deep learning models to segment bug recordings, infer user action attributes, and generate subtitle descriptions. The automated experiments demonstrate the good performance of CAPdroid in inferring user actions from the recordings, and a user study confirms the usefulness of our generated step descriptions in assisting developers with bug replay.
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{abs-2302-00886,
title = {Read It, Don't Watch It: Captioning Bug Recordings Automatically},
author = {Sidong Feng and Mulong Xie and Yinxing Xue and Chunyang Chen},
booktitle = {ICSE},
year = {2023},
doi = {10.1109/icse48619.2023.00197}
}