paperbot · PL 论文追踪

全部论文 待读 收藏

P2 TARIPlay: A Test Framework for AR Applications based on Interactive Area Detection in Playback Videos

TARIPlay:基于回放视频中交互区域检测的 AR 应用测试框架
ICSE Proceedings of the 2026 IEEE/ACM 48th International Conference on Software Engineering2026匹配度 1/10 · 该论文关注 AR 应用自动化测试框架,通过回放视频检测可交互区域,属于软件测试与计算机视觉交叉方向;与用户研究的静态分析、LLM/Agentic 程序分析、浮点程序分析或贝叶斯程序分析均无直接关联。引用 1 inbox
Seyed Amir Mousavi, Xiaoyin Wang
crossref: 2026-09-12dblp: 2026-09-12arxiv: 2026-09-12

AI 速览

一句话:提出 TARIPlay,通过分析 AR 回放视频检测、跟踪并筛选可交互区域,实现更精确的自动化 AR 应用测试。

AR 应用的交互界面来自易变且非确定的真实环境,传统 GUI 自动化测试难以直接适用。ARCore Playback 和 ARKit Replay 等回放视频技术虽可复用真实场景,但测试输入需精确计时,且视频中的交互区域动态、不规则、难以识别。为此,本文提出 TARIPlay:它分析回放视频,按稳定性和可见性等标准检测、跟踪并过滤合适的交互区域,将可测试机会交给自动化测试引擎以模拟用户交互。作者在四个开源 AR 应用和九个回放视频上评估,AR 相关代码分支覆盖率达 55.8%,显著优于 Monkey 的 41.98%,并可用于评估回放视频作为测试素材的适用性。

问题
AR 应用界面源于易变、非确定的环境,传统移动 GUI 测试方法难以直接适用;回放视频虽可复用真实场景,但测试输入需精确计时,且视频中的交互区域动态、不规则、难以识别。
方法
TARIPlay 分析回放视频,基于稳定性和可见性等标准检测、跟踪并过滤合适的交互区域,再将可测试机会输入自动化测试引擎以模拟用户交互。
评估
在四个开源 AR 应用和九个回放视频上开展实验,AR 相关代码分支覆盖率达 55.8%,显著超过现有工具 Monkey 的 41.98%,并能评估回放视频的测试适用性。
原文摘要(Abstract)

As Augmented Reality (AR) becomes more and more embedded in daily life, ensuring the quality, safety, and reliability of AR applications is increasingly important. However, AR apps present unique challenges for automated testing. Unlike static GUI layouts in traditional mobile apps, AR apps acquire their interaction interface from the surrounding environment, which is volatile and non-deterministic. Recent advancements like ARCore Playback and ARKit Replay allow developers to reuse real-world scenarios by recording and playing back enriched videos, enabling more feasible automated AR testing. However, using playback videos introduces two major challenges: test inputs must be timed precisely, and interactive areas in the video are dynamic, irregular, and difficult to identify. To address these challenges, we propose TARIPlay, a framework that analyzes playback videos to detect, track, and filter proper interactive areas over time for automated testing. In particular, TARIPlay identifies viable test opportunities based on criteria like stability and visibility, then feeds this information to an automated testing engine to simulate user interactions. We perform an experiment with four open-source AR apps and nine playback videos. Evaluation results show that TARIPlay significantly outperforms the existing tool Monkey in test coverage (55.8% over 41.98% on branch coverage) of AR-related code, and can also be used to assess the quality of playback videos for testing suitability.

链接与引用

DOI 原文 · arXiv · DBLP

BibTeX
@inproceedings{abs-2605-16544,
  title = {TARIPlay: A Test Framework for AR Applications based on Interactive Area Detection in Playback Videos},
  author = {Seyed Amir Mousavi and Xiaoyin Wang},
  booktitle = {ICSE},
  year = {2026},
  doi = {10.1145/3744916.3787817}
}