一句话：提出利用UI布局相似性的图像驱动跨平台移动测试框架LIRAT，实现平台无关的测试脚本非侵入式回放。
移动应用碎片化已从Android扩展到iOS、移动Web乃至小程序，同一应用需为不同平台分别开发和维护测试脚本，跨平台录制回放测试成本高昂。作者观察到同一应用在不同平台的用户界面布局高度相似，据此提出图像驱动的移动应用测试框架：利用计算机视觉技术对应用截图进行控件特征匹配与布局表征匹配，提取包含控件坐标、相对关系等丰富上下文信息的UI结构，据此构建平台无关的测试脚本模型，并定位待测目标控件，从而以非侵入方式回放测试脚本。作者设计并实现了工具LIRAT开展实证研究。结果表明，整体回放准确率在Android上约为65.85%（较当前最优方法提升8.74%），在iOS上为35.26%（较当前最优方法提升35%）。
The fragmentation problem has extended from Android to different platforms, such as iOS, mobile web, and even mini-programs within some applications (app), like WeChat. In such a situation, recording and replaying test scripts is one of the most popular automated mobile app testing approaches. However, such approach encounters severe problems when crossing platforms. Different versions of the same app need to be developed to support different platforms relying on different platform supports. Therefore, mobile app developers need to develop and maintain test scripts for multiple platforms aimed at completely the same test requirements, greatly increasing testing costs. However, we discover that developers adopt highly similar user interface layouts for versions of the same app on different platforms. Such a phenomenon inspires us to replay test scripts from the perspective of similar UI layouts. In this paper, we propose an image-driven mobile app testing framework, utilizing Widget Feature Matching and Layout Characterization Matching to analyze app UIs. We use computer vision (CV) technologies to perform UI feature comparison and layout hierarchy extraction on mobile app screenshots to obtain UI structures containing rich contextual information of app widgets, including coordinates, relative relationship, etc. Based on acquired UI structures, we can form a platform-independent test script, and then locate the target widgets under test. Thus, the proposed framework non-intrusively replays test scripts according to a novel platform-independent test script model. We also design and implement a tool named LIRAT to devote the proposed framework into practice, based on which, we conduct an empirical study to evaluate the effectiveness and usability of the proposed testing framework. The results show that the overall replay accuracy reaches around 65.85% on Android (8.74% improvement over state-of-the-art approaches) and 35.26% on iOS (35% improvement over state-of-the-art approaches).
DOI 原文 · arXiv · PDF（开放获取） · DBLP
@inproceedings{YuFY021,
title = {Layout and Image Recognition Driving Cross-Platform Automated Mobile Testing},
author = {Shengcheng Yu and Chunrong Fang and Yexiao Yun and Yang Feng},
booktitle = {ICSE},
year = {2021},
doi = {10.1109/icse43902.2021.00139}
}