一句话:提出 TARIPlay,通过分析 AR 回放视频检测、跟踪并筛选可交互区域,实现更精确的自动化 AR 应用测试。
AR 应用的交互界面来自易变且非确定的真实环境,传统 GUI 自动化测试难以直接适用。ARCore Playback 和 ARKit Replay 等回放视频技术虽可复用真实场景,但测试输入需精确计时,且视频中的交互区域动态、不规则、难以识别。为此,本文提出 TARIPlay:它分析回放视频,按稳定性和可见性等标准检测、跟踪并过滤合适的交互区域,将可测试机会交给自动化测试引擎以模拟用户交互。作者在四个开源 AR 应用和九个回放视频上评估,AR 相关代码分支覆盖率达 55.8%,显著优于 Monkey 的 41.98%,并可用于评估回放视频作为测试素材的适用性。
As Augmented Reality (AR) becomes more and more embedded in daily life, ensuring the quality, safety, and reliability of AR applications is increasingly important. However, AR apps present unique challenges for automated testing. Unlike static GUI layouts in traditional mobile apps, AR apps acquire their interaction interface from the surrounding environment, which is volatile and non-deterministic. Recent advancements like ARCore Playback and ARKit Replay allow developers to reuse real-world scenarios by recording and playing back enriched videos, enabling more feasible automated AR testing. However, using playback videos introduces two major challenges: test inputs must be timed precisely, and interactive areas in the video are dynamic, irregular, and difficult to identify. To address these challenges, we propose TARIPlay, a framework that analyzes playback videos to detect, track, and filter proper interactive areas over time for automated testing. In particular, TARIPlay identifies viable test opportunities based on criteria like stability and visibility, then feeds this information to an automated testing engine to simulate user interactions. We perform an experiment with four open-source AR apps and nine playback videos. Evaluation results show that TARIPlay significantly outperforms the existing tool Monkey in test coverage (55.8% over 41.98% on branch coverage) of AR-related code, and can also be used to assess the quality of playback videos for testing suitability.