一句话：通过314个开源VR应用的实证研究，揭示VR自动化测试缺失、覆盖率工具不足与断言密度低等现状。
虚拟现实（VR）应用需要物理交互并依赖特定硬件，其自动化测试困难，而业界对VR应用的测试实践了解甚少。本文对314个开源VR应用开展实证研究，分析自动测试存在性、功能方法与测试方法比例、VR代码覆盖率工具支持、断言密度，并人工分析370个测试用例以归纳测试类别。结果显示：79%的VR项目没有任何自动测试；有测试的项目其功能-测试方法比中位数低于其他项目；VR代码覆盖率测量工具支持存在问题，断言密度平均仅17.63%；研究还识别出用于验证VR质量属性的测试用例类别，并区分出需要特别关注VR特性的类别。最后呼吁社区改进VR自动化测试实践。
Virtual Reality (VR) is an emerging technique that provides a unique real-time experience for users. VR technologies have provided revolutionary user experiences in various scenarios (e.g., training, education, gaming, etc.). However, testing VR applications is challenging due to their nature which necessitates physical interactivity, and their reliance on specific hardware systems. Despite the recent advancements in VR technology and its usage scenarios, we still know little about VR application testing. To fill up this knowledge gap, we performed an empirical study on 314 open-source VR applications. Our analysis identified that 79% of the VR projects evaluated did not have any automatic tests, and for the VR projects that did, the median functional-method to test-method ratios were lower than those of other project types. Moreover, we uncovered tool support issues concerning the measurement of VR code coverage, and the assertion density results we were able to generate were relatively low, with an average of 17.63%. Finally, through a manual analysis of 370 test cases, we identified the different categories of test cases being used to validate VR application quality attributes. Furthermore, we extracted which of these categories are VR-attention, meaning that test writers need to pay special attention to VR characteristics when writing tests of these categories. We believe that our findings constitute a call to action for the VR development community to improve their automatic testing practices and provide directions for software engineering researchers to develop advanced techniques for automatic test case generation and test quality analysis for VR applications. Our replication package containing the dataset we used, software tools we developed, and the results we found, is accessible at https://doi.org/10.6084/m9.figshare.19678938.
@inproceedings{RzigIAQH23,
title = {Virtual Reality (VR) Automated Testing in the Wild: A Case Study on Unity-Based VR Applications},
author = {Dhia Elhaq Rzig and Nafees Iqbal and Isabella Attisano and Xue Qin and Foyzul Hassan},
booktitle = {ISSTA},
year = {2023},
doi = {10.1145/3597926.3598134}
}