一句话：提出 VOPA，利用众包标注与混合神经网络自动预测 AR 测试预言，检测虚拟对象放置不精确问题。
增强现实（AR）应用常因虚拟对象放置不精确而引发可用性问题，但自动检测这类缺陷极具挑战，因为其依赖人类感知与验证。本文提出 VOPA（虚拟对象放置评估），通过插桩真实 AR 应用，在真实场景中收集屏幕截图与对应元数据，再经众包标注后训练一个混合神经网络来识别对象放置错误。VOPA 旨在自动化评估虚拟对象放置质量并检测不精确实例，从而增强 AR 应用测试。在 304 张截图的测试集上，VOPA 达到 99.34% 的准确率、96.92% 的精确率和 100% 的召回率；此外还成功识别出 38 个真实世界的放置错误，包括对象悬空于两个表面之间或嵌入墙体等情形。
Augmented Reality (AR) technology opens up exciting possibilities in various fields, such as education, work guidance, shopping, communication, and gaming. However, users often encounter usability and user experience issues in current AR apps, often due to the imprecise placement of virtual objects. Detecting these inaccuracies is crucial for AR app testing, but automating the process is challenging due to its reliance on human perception and validation. This paper introduces VOPA (Virtual Object Placement Assessment), a novel approach that automatically identifies imprecise virtual object placements in real-world AR apps. VOPA involves instrumenting real-world AR apps to collect screenshots representing various object placement scenarios and their corresponding metadata under real-world scenes. The collected data are then labeled through crowdsourcing and used to train a hybrid neural network that identifies object placement errors. VOPA aims to enhance AR app testing by automating the assessment of virtual object placement quality and detecting imprecise instances. In our evaluation of a test set of 304 screenshots, VOPA achieved an accuracy of 99.34%, precision of 96.92% and recall of 100%. Furthermore, VOPA successfully identified 38 real-world object placement errors, including instances where objects were hovering between two surfaces or appearing embedded in the wall.
@inproceedings{paperbot8549,
title = {Towards Automatic Oracle Prediction for AR Testing: Assessing Virtual Object Placement Quality under Real-World Scenes},
author = {Xiaoyi Yang and Yuxing Wang and Tahmid Rafi and Dongfang Liu and Xiaoyin Wang and Xueling Zhang},
booktitle = {ISSTA},
year = {2024},
doi = {10.1145/3650212.3680315}
}